Move with the theremin: Body Posture and Gesture Recognition using the Theremin in Loose-Garment with Embedded Textile Cables as Antennas

Hymalai Bello, Bo Zhou, Sungho Suh, Luis Sanchez, Paul Lukowicz

In: Xianta Jiang (editor). Frontiers in Computer Science (Front. Comput. Sci.) Mobile and Ubiquitous Computing Volume 4 Pages 1-15 Frontiers 6/2022.


We present a novel intelligent garment design approach for body posture/gesture detection in the form of a loose-fitting blazer prototype, “the MoCaBlazer.” The design is realized by leveraging conductive textile antennas with the capacitive sensing modality, supported by an open-source electronic theremin system (OpenTheremin). The use of soft textile antennas as the sensing element allows flexible garment design and seamless tech-garment integration for the specific structure of different clothes. Our novel approach is evaluated through two experiments involving defined movements (20 arm/torso gestures and eight dance movements). In cross-validation, the classification model yields up to 97.18% average accuracy and 92% f1-score, respectively. We have also explored real-time inference enabled by a radio frequency identification (RFID) synchronization method, yielding an f1-score of 82%. Our approach opens a new paradigm for designing motion-aware smart garments with soft conductive textiles beyond traditional approaches that rely on tight-fitting flexible sensors or rigid motion sensor accessories.


Weitere Links

fcomp-04-915280.pdf (pdf, 4 MB )

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz