Publication
Breaking the 3D Dataset Bottleneck: Fast Scalable Generation of Aligned 3D Assets from Scratch for Category 6D Pose Estimation and Robotic Grasping
Guillaume Duret; Danylo Mazurak; Florence Zara; Jan Peters; Liming Chen
In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2026, Denver, CO, USA, June 3-7, 2026. International Conference on Computer Vision and Pattern Recognition (CVPR), Pages 1809-1818, Computer Vision Foundation, 2026.
Abstract
While 2D vision has been revolutionized by large-scale
datasets, 3D vision remains constrained by scarce, canon-
ically aligned data. We introduce the first scalable, au-
tomated framework that generates complete category-level
6D pose datasets directly from text prompts, bypassing ex-
isting 3D assets. Our method achieves: (1) reliable asset
generation via a controlled text-to-image-to-3D pipeline;
(2) built-in canonical alignment through depth-conditioned
generation (96% pose consistency); (3) large-scale 6D an-
notation via mixed reality rendering. The pipeline pro-
duces aligned meshes in under 3 minutes per object (5–20×
speedup). We generate over 1,000 instances for each of
153 categories (153,000 meshes, ¿40× increase per cat-
egory). Extensive evaluation shows competitive zero-shot
sim2real transfer on NOCS and superior robotic grasp-
ing (87.8% success), where aligned meshes prove essen-
tial. We release the largest publicly available aligned 3D
mesh dataset, category-level 6D pose dataset, grasping en-
vironments, and open-source pipeline.