CoMeSy

Student Assistant Project | University of Applied Sciences Fulda | Human-Robot Interaction

CoMeSy is an interdisciplinary research project at the University of Applied Sciences Fulda in collaboration with an industry partner. The goal is to enable intuitive and safe control of a UR5 robotic arm – even for users without programming experience – using gesture recognition and visual feedback systems.

I’m part of the development team and work as a student assistant on two specific subprojects:

VR/AR-based UR5 Robot Control

In this part of the project, I’m developing a VR application in Unity3D for the Oculus Quest 3 that places a realistically scaled URDF model of the UR5 robotic arm into the environment using AR.

This virtual model can be controlled via the same ROS2 topics and a ROS# bridge as the real robot.
The goal is to allow testing of the control logic outside the lab – for example during development or demonstrations.

In the future, the system will be expanded to support a shared AR view for multiple headsets, enabling collaborative presentations or training scenarios.

Tabletop Projection for Human-Robot Communication

In a second subproject, I’m building a visualization system that projects movement and status data from the UR5 robot directly onto the workspace – using a beamer, along with audio output.

The system includes:

• a FastAPI server for managing data,
• a REST API for external integration,
• and a PyGame-based visualizer that renders warnings, planned movements, and spoken cues in real time.

A key challenge was ensuring smooth, real-time visualization while keeping the architecture modular and future-proof – for example to support a potential web interface later on.