Publikation
GaussTwin: Unified Simulation and Correction with Gaussian Splatting for Robotic Digital Twins
Yichen Cai; Paul Jansonnie; Cristiana de Farias; Oleg Arenz; Jan Peters
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2603.05108, Pages 1-8, arXiv, 2026.
Zusammenfassung
Digital twins promise to enhance robotic manipu-
lation by maintaining a consistent link between real-world per-
ception and simulation. However, most existing systems struggle
with the lack of a unified model, complex dynamic interactions,
and the real-to-sim gap, which limits downstream applications
such as model predictive control. Thus, we propose GaussTwin,
a real-time digital twin that combines position-based dynamics
with discrete Cosserat rod formulations for physically grounded
simulation, and Gaussian splatting for efficient rendering and
visual correction. By anchoring Gaussians to physical primitives
and enforcing coherent SE(3) updates driven by photometric
error and segmentation masks, GaussTwin achieves stable pre-
diction–correction while preserving physical fidelity. Through
experiments in both simulation and on a Franka Research
3 platform, we show that GaussTwin consistently improves
tracking accuracy and robustness compared to shape-matching
and rigid-only baselines, while also enabling downstream tasks
such as push-based planning. These results highlight GaussTwin
as a step toward unified, physically meaningful digital twins
that can support closed-loop robotic interaction and learning.