Development of a modular software architecture for controlling a 7-axis robotic arm – with 3D object recognition, gesture recognition, and voice control for a collaborative research robot.
Client/Company/Industry
IAV GmbH
Duration
24 months
Product
Software
Expertise
Software Development
On behalf of a leading automotive development services provider, a software architecture was to be developed that enables control of a 7-axis Franka robotic arm through three input channels: 3D object recognition, gesture recognition, and voice control.
A core requirement was a fully reactive system capable of responding to new inputs at any point in time - even while the robot is already executing an action. The solution was to be built on ROS and delivered as a containerised development infrastructure.
The greatest challenge was real-time reactivity: every ongoing action of the robotic arm had to be interruptible and replaceable by a new input at any moment. A framework based on asynchronously executable functions was developed to ensure full system responsiveness at all times.
The 3D object recognition methods proved sensitive to environmental conditions and model parameters. A systematic methodology for parameter tuning was developed to minimise recurring recognition errors and improve robustness in operation.
Programming Languages
Python, C++
Technologies
ROS, MoveIt!, OpenCV, PCL, TensorFlow, Docker, Qt
Visualisation of the robotic arm control interface with 3D point cloud view and motion planning in RViz.
Similar problem?
The result is a modular, extensible robot control architecture that translates multimodal inputs - objects, gestures, and voice - into coordinated motion sequences of a collaborative robotic arm. The asynchronous communication model proved to be a solid foundation for reactive robotics applications in research environments.
RIM2D is an existing, highly efficient 2D hydraulic simulation model for fluvial, pluvial, and urban flooding. As part of a strategic partnership, we supported the extension of the research code with a web application and a cloud-based GPU simulation environment, enabling its transition into a market-ready product.
We developed an Open-Source S3-based data lake solution for the centralized ingestion, categorization, and searchability of data. The goal was to automate and improve manual data management through an integrated architecture with workflow orchestration, data cataloging, and access control.