Dynamical motions, like hopping, enhance the locomotion potential of legged robots. To address the complexity of controlling robot hopping, the researchers introduce an innovative approach in their paper titled “End-to-End Reinforcement Learning for Torque-Based Variable Height Hopping”. This involves the use of reinforcement learning to accurately detect hopping phases and translate them into torque-based control.
Trajectory planning is widely used in underactuated robot motion control. Yet, when used with series-parallel hybrid kinematics in humanoid robots, it neglects closed-loop controls, limiting the system's full potential. In the paper “Exploiting a Series-Parallel Hybrid Humanoid's Full Capabilities via Whole Body Trajectory Optimization” researchers showcase trajectory optimization for a series-parallel hybrid robot, considering holonomic constraints arising from closed kinematic loops in a weightlifting task.
The paper “AcroMonk: A Minimalist Underactuated Brachiating Robot” presents a simple brachiation robot prototype with only one actuator and passive grippers for monkey-like movements. The design uses different control methods, such as trajectory optimization and reinforcement learning, comparing their effectiveness and energy efficiency. The open-sourced design offers a valuable educational and research resource for underactuated robotics.
Introduced for serial robotic arms, series elastic actuators (SEA) require algorithms for model-based trajectory tracking control that involve changes over time. However, this approach has not been explored with SEAs for parallel kinematics manipulators (PKM). The paper “A Recursive Lie-Group Formulation for the Second-Order Time Derivatives of the Inverse Dynamics of Parallel Kinematic Manipulators” bridges this gap by extending the formulation for PKMs, adapting techniques from simpler robots using a so-called Lie group framework.
The accepted papers in detail:
Title: End-to-End Reinforcement Learning for Torque Based Variable Height Hopping
Authors: Raghav Soni, Daniel Harnack, Hannah Isermann, Sotaro Fushimi, Shivesh Kumar, Frank Kirchner
Abstract:
Legged locomotion is arguably the most suited and versatile mode to deal with natural or unstructured terrains. Intensive research into dynamic walking and running controllers has recently yielded great advances, both in the optimal control and reinforcement learning (RL) literature. Hopping is a challenging dynamic task involving a flight phase and has the potential to increase the traversability of legged robots. Model based control for hopping typically relies on accurate detection of different jump phases, such as lift-off or touch down, and using different controllers for each phase. In this paper, we present an end-to-end RL based torque controller that learns to implicitly detect the relevant jump phases, removing the need to provide manual heuristics for state detection. We also extend a method for simulation to reality transfer of the learned controller to contact rich dynamic tasks, resulting in successful deployment on the robot after training without parameter tuning.
Paper Link: https://arxiv.org/pdf/2307.16676.pdf
Video Link: End-to-End Reinforcement Learning for Torque Based Variable Height Hopping
Title: Investigations into Exploiting the Full Capabilities of a Series-Parallel Hybrid Humanoid Using Whole Body Trajectory Optimization
Authors: Melya Boukheddimi, Rohit Kumar, Shivesh Kumar, Justin Carpentier, Frank Kirchner
Abstract:
Trajectory optimization methods have become ubiquitous for the motion planning and control of under-actuated robots for e.g., quadrupeds, humanoids etc. While they have been extensively used in the case of serial or tree type robots, they are seldomly used for planning and control of robots with closed loops. Series-parallel hybrid topology is quite commonly used in the design of humanoid robots, but they are often neglected during trajectory optimization and the movements are computed for a serial abstraction of the system and then the solution is mapped to the actuator coordinates. As a consequence, the full capability of the robot cannot be exploited. This paper presents a case study of trajectory optimization for series-parallel hybrid robot by taking into account all the holonomic constraints imposed by the closed kinematic loops present in the system. We demonstrate the advantages of this consideration with a weightlifting task on RH5 Manus humanoid in both simulation and experiments.
Paper Link: https://www.researchgate.net/publication/372677906_Investigations_into_Exploiting_the_Full_Capabilities_of_a_Series-Parallel_Hybrid_Humanoid_using_Whole_Body_Trajectory_Optimization
Video Link: https://www.youtube.com/watch?v=qcJiLLTbDmk
Title: AcroMonk: A Minimalist Underactuated Brachiating Robot
Authors: Mahdi Javadi, Daniel Harnack, Paula Stocco, Shivesh Kumar, Shubham Vyas, Daniel Pizzutilo, Frank Kirchner
Abstract:
Brachiation is a dynamic, coordinated swinging maneuver of body and arms used by monkeys and apes to move between branches. As a unique underactuated mode of locomotion, it is interesting to study from a robotics perspective since it can broaden the deployment scenarios for humanoids and animaloids. While several brachiating robots of varying complexity have been proposed in the past, this paper presents the simplest possible prototype of a brachiation robot, using only a single actuator and unactuated grippers. The novel passive gripper design allows it to snap on and release from monkey bars, while guaranteeing well defined start and end poses of the swing. The brachiation behavior is realized in three different ways, using trajectory optimization via direct collocation and stabilization by a model-based time-varying linear quadratic regulator (TVLQR) or model-free proportional derivative (PD) control, as well as by a reinforcement learning (RL) based control policy. The three control schemes are compared in terms of robustness to disturbances, mass uncertainty, and energy consumption. The system design and controllers have been open-sourced. Due to its minimal and open design, the system can serve as a canonical underactuated platform for education and research.
Paper Link: https://arxiv.org/pdf/2305.08373.pdf
Video Link: https://www.youtube.com/watch?v=FIcDNtJo9Jc
GitHub Link: https://github.com/dfki-ric-underactuated-lab/acromonk
Title: A Recursive Lie-Group Formulation for the Second-Order Time Derivatives of the Inverse Dynamics of Parallel Kinematic Manipulators
Authors: Andreas Mueller, Shivesh Kumar, Thomas Kordik
Abstract:
Series elastic actuators (SEA) were introduced for serial robotic arms. Their model-based trajectory tracking control requires the second time derivatives of the inverse dynamics' solution, for which algorithms were proposed. Trajectory control of parallel kinematics manipulators (PKM) equipped with SEAs has not yet been pursued. Key element for this is the computationally efficient evaluation of the second time derivative of the inverse dynamics' solution. This has not been presented in the literature and is addressed in the present paper for the first time. The special topology of PKM is exploited reusing the recursive algorithms for evaluating the inverse dynamics of serial robots. A Lie group formulation is used, and all relations are derived within this framework. Numerical results are presented for a 6-DOF Gough-Stewart platform (as part of an exoskeleton), and for a planar PKM when a flatness-based control scheme is applied.
Paper Link (open access): https://ieeexplore.ieee.org/document/10101821