An assistive robot arm is a valuable support for people with severely reduced mobility of arm and hand, but exhausting and time-consuming to use, because the seven movement possibilities (degrees of freedom) of the robot arm are only opposed by two of a typical input device (e.g., joystick). Therefore, it is very often necessary to switch manually between movement possibilities. The aim of the project is to facilitate the use of assistive robotic arms by combining sensor-based situational recognition with algorithm-based control to create a learning AI-based control system. The movement possibilities proposed by the system help users through a simplified control of the robotic assistance system, thus increasing their autonomy in their daily lives.
DFKI is developing the shared control of the Jaco robotic arm, based on the implementation of a Convolutional Neural Network (CNN) architecture that maps depth images from two TOF cameras and the current 6D pose of the robot arm on two main freedom devices for control with the help of a 2D input device. For the CNN training, the DFKI builds a real-world and a simulation training system in which instructed subjects demonstrate robot-arm movements for ADLs with a 6 DoF SpaceMouse. Together with Munevo GmbH and Kinova (associated partner), DFKI integrates the hardware and software components of the Kinova JACO into the autonomous wheelchair Rolland (Otto Bock Xeno basis) for the demonstrator. These include the electronic and sensory integration, as well as the adaptation of the driver assistance software.
Partners
- Frankfurt University of Applied Sciences, Frankfurt
- Westfälische Hochschule Gelsenkirchen Bocholt Recklinghausen, Gelsenkirchen
- Munevo GmbH, München
Prof. Dr. Udo Frese
