Publication
More Structure, Not More Capacity: Object-Centric Representations for Visuomotor Imitation Learning
Yi Li; Alexandre Chapin; Liming Chen; Jan Peters; Alap Kshirsagar
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2607.09825, Pages 1-7, arXiv, 2026.
Abstract
Robotic manipulation policies rely on pre-trained
vision models that give either a global scene embedding or a dense
patch grid. Both mix task-relevant and task-irrelevant features.
Object-centric slot representations are a structured alternative:
they group features into a few per-object slots. We test what this
structure buys on ManiSkill3 PickCube-v1, with a frozen encoder
and a held-out-seed evaluation. Holding the policy, goal token,
rendering, and calibration fixed and changing only the encoder, a
frozen object-centric SPOT representation (DINO ViT-B/16 + Slot
Attention) reaches 55.0±2.9% success, 22.4% above a dense DINO
global-feature baseline (32.6 ± 1.5%), with the same trainable
policy and no encoder fine-tuning. More tokens alone do not
help: a dense patch grid with 16× the tokens performs no better
than the global feature. Adding an explicit 2D spatial goal and
native-resolution rendering raises the full system to 68.7±4.2%,
just below a privileged 3D-oracle upper bound (71.7±4.1%). An
automated kinematic failure taxonomy then separates spatial-
precision (Near-Miss) failures from object-tracking (No-Grasp)
failures: spatial grounding reduces Near-Miss while leaving No-
Grasp unchanged. The same taxonomy transfers to the harder
StackCube-v1 and points to occlusion as the main bottleneck.