Publikation
Learning Multimodal Latent Dynamics for Human-Robot Interaction
Vignesh Prasad; Lea Heitlinger; Dorothea Koert; Ruth Stock-Homburg; Jan Peters; Georgia Chalvatzaki
In: IEEE Transactions on Robotics (T-RO), Vol. 41, Pages 4418-4438, arXiv, 2025.
Zusammenfassung
This article presents a method for learning well-
coordinated Human-Robot Interaction (HRI) from Human-
Human Interactions (HHI). We devise a hybrid approach using
Hidden Markov Models (HMMs) as the latent space priors for
a Variational Autoencoder to model a joint distribution over
the interacting agents. We leverage the interaction dynamics
learned from HHI to learn HRI and incorporate the conditional
generation of robot motions from human observations into the
training, thereby predicting more accurate robot trajectories.
The generated robot motions are further adapted with Inverse
Kinematics to ensure the desired physical proximity with a
human, combining the ease of joint space learning and accu-
rate task space reachability. For contact-rich interactions, we
modulate the robot’s stiffness using HMM segmentation for a
compliant interaction. We verify the effectiveness of our approach
deployed on a Humanoid robot via a user study. Our method
generalizes well to various humans despite being trained on
data from just two humans. We find that users perceive our
method as more human-like, timely, and accurate and rank our
method with a higher degree of preference over other baselines.
We additionally show the ability of our approach to generate
successful interactions in a more complex scenario of Bimanual
Robot-to-Human Handovers.