Publication
Qualitative Comparison Between Marker-Based and Video-Based Human Pose Estimation in the Context of Imitation Learning
Max Lödige; Alexander Fabisch; Lisa Gutzeit
In: Diedrich Wolter; Gesina Schwalbe (Hrsg.). KI 2026: Advances in Artificial Intelligence. German Conference on Artificial Intelligence (KI-2026), 49th German Conference on Artificial Intelligence, Cham, Pages 260-268, ISBN 978-3-032-32335-4, Springer Nature Switzerland, 2027.
Abstract
Imitation learning from human demonstration often relies on marker-based motion capture, which restricts data recording to controlled environments. Video-based motion capture presents an alternative that reduces the recording effort. To compare both, we develop a novel evaluation framework using four categories of metrics, namely pose estimation quality, trajectory distance, behaviour separability, and grasp characterization, applied to three state-of-the-art video-based 3D pose estimation models in a pick-and-place scenario with parallel recording.
Using this framework, we show that video-based pose estimation using a depth map captures actions and trajectories well enough for imitation learning, but high-fidelity movement such as grasping remains beyond the current capabilities of all evaluated video-based modalities.