Publikation
Hardness Similarity Detection Using Vision-Based Tactile Sensors
Alap Kshirsagar; Frederik Heller; Mario Gomez Andreu; Boris Belousov; Tim Schneider; Lisa P. Y. Lin; Katja Doerschner; Knut Drewing; Jan Peters
In: 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40). IEEE International Conference on Robotics and Automation (ICRA-2024), September 23-26, Rotterdam, Netherlands, IEEE, 2024.
Zusammenfassung
Humans can classify deformable materials accord-
ing to their hardness similarity, but existing robotic approaches
focus on hardness recognition or absolute hardness prediction.
In this work, we investigate hardness similarity detection
using a vision-based tactile sensor (VBTS) and evaluate three
methods: optical flow features and support vector machine
(SVM) classifier, DINOv2 features and SVM classifier, and
convolutional long short term memory (ConvLSTM) network
trained with categorical cross-entropy loss. To evaluate these
methods, we created a dataset of over 200 videos by pressing
a GelSight Mini sensor, attached to a Franka-Panda robot, on
five silicone objects of varying hardness, and also conducted
a human-participant study showing humans achieved 80.25%
average accuracy in hardness similarity detection. The three
methods achieved average accuracies of 77.66%, 67.00%, and
70.00% with 15 samples per object, demonstrating that a VBTS
can effectively classify objects based on hardness similarity.