Machine learning and, in particular, reinforcement learning methods are now considered a key technology for any area of robotic learning with application potential in both terrestrial and space scenarios. However, these methods are extremely complex in their architecture and require a significant amount of training steps, making learning new challenging behaviors in real robotic environments nearly impossible without prior knowledge and simulation environment. Quantum algorithms, which hold the potential to process and analyze large amounts of data much more efficiently than classical machine learning algorithms, could remedy this situation. By reducing training times, among other things, they promise an immense leap in terms of the complexity of use cases, which could lead to significant progress in the field of long-term learning of exploratory robotic systems.
Basic research on quantum computing and quantum machine learning
However, research into quantum technology in this area is still in its infancy. Scientists at the DFKI Robotics Innovation Center and the AG Robotik at the University of Bremen have set their sights on advancing this research. Since 2020, the interdisciplinary team led by Prof. Dr. Dr. h.c. Frank Kirchner has been conducting fundamental research to develop quantum-based concepts and solutions for application fields in artificial intelligence and robotics. By developing demand-oriented and low-threshold qualification modules, they also want to contribute to counteracting the shortage of skilled workers in the technology field of quantum-based artificial intelligence.
With the projects QuDA-KI, QuBER-KI and QuMAL-KI, the German Federal Ministry of Economics and Climate Protection (BMWK) is currently funding three synergetic projects that aim to both evaluate and improve existing methods of quantum machine-based learning and develop new methods for robotic applications. In addition to purely quantum-based approaches, the methods under investigation include hybrid methods in which certain portions of the algorithm are offloaded to quantum computers while the remaining portion is processed on a classical computer.
Encoding classical datasets into qubits and building a quantum computing lab
In order to use robotic data streams, especially from sensors and actuators, for quantum machine learning, they must be in suitable qubit representations. The Bremen researchers are investigating how the data can be encoded in the QuDA-KI project (qubit-based data representations and preprocessing for quantum machine learning approaches). The focus is on qubit-based minimal representations of essential features in order to be able to implement first use cases with the few qubits available in today's quantum computers. In addition, data sets from robotic scenarios of previous DFKI work will be processed for use on quantum hardware and made available to other projects. The construction of a laboratory equipped with powerful hardware and software required for the simulation of quantum-assisted processes is also part of the project. In addition, the researchers in the project want to figure out how to make quantum circuits more efficient.
Quantum-assisted reinforcement learning techniques for simple robot behaviors
The findings of QuDA-KI will flow into the two application-oriented projects QuBER-KI and QuMAL-KI, in which quantum-assisted reinforcement learning methods will be used to generate concrete robot behavior. In QuBER-KI (Quantum Deep Reinforcement Learning for simple robotic behaviors), the scientists analyze existing quantum-assisted algorithms and evaluate them with respect to the question whether and to what extent they can be transferred to more complex environments and applications. Furthermore, the neural networks of the classical algorithms currently successfully used in practice for reinforcement learning will be replaced by quantum circuits in order to determine which algorithms achieve the best results for which learning problems. Based on this, the researchers aim to develop new quantum-assisted reinforcement learning methods for simple robotic behaviors such as navigation or manipulation that explicitly exploit quantum mechanical properties.
Quantum-accelerated learning for multi-agent systems and coordinative problems
Especially for challenging scenarios in space robotics, e.g., planetary exploration or infrastructure construction, future long-term autonomous robots must be able to learn and, if necessary, adapt complex behaviors in interaction with other robotic systems. The QuMAL-KI (Quantum Accelerated Multi-Agent Learning for Long-Term Autonomous Robots) project aims to accelerate reinforcement-based machine learning procedures for both multiple robotic systems and multiple quantum computers to optimize or learn a particular behavior in a distributed setting. To this end, the researchers will first evaluate existing quantum algorithms and then develop new methods to be tested in a simple but realistic scenario with at least two robots. In all three projects, the majority of the algorithms will be executed in simulation. In addition, it is planned to demonstrate certain parts of the project on currently available quantum hardware.
The projects QuDA-KI (duration: 01.10.2022 to 30.09.2025), QuBER-KI (duration: 01.11.2022 to 31.10.2025) and QuMAL-KI (duration: 01.12.2022 to 30.11. 2026) are funded by the German Federal Ministry of Economics and Climate Protection (BMWK) via the German Aerospace Center under the funding codes 50RA2206A and 50RA2206B (QuDA-KI), 50RA2207A and 50RA2207B (QuBER-KI), 50RA2208A and 50RA2208B (QuMAL-KI).
