Skip to main content Skip to main navigation

Publication

CaptAinGlove: Capacitive and Inertial Fusion-Based Glove for Real-Time on Edge Hand Gesture Recognition for Drone Control

Hymalai Bello; Sungho Suh; Daniel Geißler; Lala Shakti Swarup Ray; Bo Zhou; Paul Lukowicz
In: Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing. International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp-2023), located at International Symposium on Wearable Computing, October 8-12, Cancun, Mexico, Mexico, UbiComp/ISWC '23 Adjunct, ISBN 9798400702006, Association for Computing Machinery, 10/2023.

Abstract

We present CaptAinGlove, a textile-based, low-power (≤ 1.15Watts), privacy-conscious, real-time on-the-edge (RTE) glove-based solution with a tiny memory footprint (≤ 2MB), designed to recognize hand gestures used for drone control. We employ lightweight convolutional neural networks as the backbone models and a hierarchical multimodal fusion to reduce power consumption and improve accuracy. The system yields an F1-score of 80% for the offline evaluation of nine classes; eight hand gesture commands and null activity. For the RTE, we obtained an F1-score of 67% (one user).

Projects

More links