Skip to main content Skip to main navigation

Publikation

Learning Force Distribution Estimation for the GelSight Mini Optical Tactile Sensor Based on Finite Element Analysis

E. Helmut; L. Dziarski; N. Funk; B. Belousov; Jan Peters
In: 2nd NeurIPS Workshop on Touch Processing: From Data to Knowledge. Neural Information Processing Systems (NeurIPS-2024), NeurIPS, 2024.

Zusammenfassung

Contact-rich manipulation remains a major challenge in robotics. Optical tactile sensors like GelSight Mini offer a low-cost solution for contact sensing by capturing softbody deformations of the silicone gel. However, accurately inferring shear and normal force distributions from these gel deformations has yet to be fully addressed. In this work, we propose a machine learning approach using a U-net architecture to predict force distributions directly from the sensor’s raw images. Our model, trained on force distributions inferred from Finite Element Analysis (FEA), demonstrates promising accuracy in predicting normal and shear force distributions. It also shows potential for generalization across sensors of the same type and for enabling real-time application. The codebase, dataset and models are open-sourced and available at https://feats-ai.github.io.