Skip to main content Skip to main navigation

Publikation

Hardness Similarity Detection Using Vision-Based Tactile Sensors

Alap Kshirsagar; Frederik Heller; Mario Gomez Andreu; Boris Belousov; Tim Schneider; Lisa P. Y. Lin; Katja Doerschner; Knut Drewing; Jan Peters
In: 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40). IEEE International Conference on Robotics and Automation (ICRA-2024), September 23-26, Rotterdam, Netherlands, IEEE, 2024.

Zusammenfassung

Humans can classify deformable materials accord- ing to their hardness similarity, but existing robotic approaches focus on hardness recognition or absolute hardness prediction. In this work, we investigate hardness similarity detection using a vision-based tactile sensor (VBTS) and evaluate three methods: optical flow features and support vector machine (SVM) classifier, DINOv2 features and SVM classifier, and convolutional long short term memory (ConvLSTM) network trained with categorical cross-entropy loss. To evaluate these methods, we created a dataset of over 200 videos by pressing a GelSight Mini sensor, attached to a Franka-Panda robot, on five silicone objects of varying hardness, and also conducted a human-participant study showing humans achieved 80.25% average accuracy in hardness similarity detection. The three methods achieved average accuracies of 77.66%, 67.00%, and 70.00% with 15 samples per object, demonstrating that a VBTS can effectively classify objects based on hardness similarity.