Publication
SynthoGestures: A Novel Framework for Synthetic Dynamic Hand Gesture Generation for Driving Scenarios
Amr Gomaa; Robin Zitt; Guillermo Reyes; Antonio Krüger
In: Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM Symposium on User Interface Software and Technology (UIST-2023), New York, NY, USA, UIST '23 Adjunct, ISBN 9798400700965, Association for Computing Machinery (ACM), 10/2023.
Abstract
Creating a diverse and comprehensive dataset of hand gestures for dynamic human-machine interfaces in the automotive domain can be challenging and time-consuming. To overcome this challenge, we propose using synthetic hand gestures generated by virtual 3D models. In this paper, we present our open-source framework that utilizes Unreal Engine to synthesize realistic static and dynamic hand gestures, offering customization options and reducing the risk of overfitting. Multiple variants, including gesture speed, performance, and hand shape, are generated to improve generalizability. In addition, we simulate different camera locations and types, such as RGB, infrared, and depth cameras, without incurring additional time, effort, or cost to obtain these cameras. Experimental results demonstrate that our proposed framework, SynthoGestures, improves gesture recognition accuracy and can replace or augment real-hand datasets. By saving time and effort in the creation of a data set, our tool accelerates the development of gesture recognition systems for automotive and non-automotive applications.
Projects
- CAMELOT - Continuous Adaptive Machine-Learning of Transfer of Control Situations
- SC_Gomaa - TeachTAM: Machine Teaching with Hybrid Neurosymbolic Reinforcement Learning; The Apprenticeship Model