Every9D-21M: Large-Scale Real-World 9D Canonicalization of Everyday Objects

2026-05-27 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Data Science & Analytics · Depth: Expert, medium

Summary

Every9D-21M is a new large-scale dataset designed to address the challenge of 9D pose estimation for everyday objects from single real-world images, a task previously hindered by insufficient supervision. This dataset comprises 21.8 million real-world images derived from 109,000 object-centric videos, covering 700 distinct everyday object categories. This scale represents a two orders of magnitude increase over previous real-world 9D pose benchmarks in both image and category count. The creation method involves reconstructing object-level point clouds using multi-view geometry and aligning similar instances into a shared canonical coordinate frame. Canonical poses are manually annotated for less than 0.01% of images and then propagated and verified across the remaining instances. The dataset also includes cross-category orientation rules for symmetry-aware evaluation. Training on Every9D-21M significantly improves performance on ImageNet3D and PASCAL3D+, and demonstrates superior generalization to HANDAL compared to ImageNet3D.

Key takeaway

For AI Scientists developing 9D pose estimation models, you should consider Every9D-21M as a foundational training resource. Its unprecedented scale of 21.8 million real-world images across 700 categories offers superior generalization capabilities compared to smaller, synthetic datasets. Integrating this dataset can significantly improve your model's performance on real-world benchmarks like ImageNet3D and PASCAL3D+, and enhance generalization to novel environments such as HANDAL. Explore the provided data and code to accelerate your research.

Key insights

Large-scale, real-world 9D pose datasets can be built efficiently by propagating sparse manual annotations across object-centric videos.

Principles

• Multi-view geometry enables object-level point cloud reconstruction.
• Cross-instance alignment propagates canonical poses effectively.
• Cross-category rules induce symmetry for robust evaluation.

Method

Reconstruct point clouds from object-centric videos, align instances to a shared canonical frame, manually annotate a small reference set, propagate poses, and verify.

In practice

• Use object-centric videos for scalable data generation.
• Implement cross-instance alignment for pose propagation.
• Apply cross-category rules for symmetry-aware evaluation.

Topics

• 9D Pose Estimation
• Object Canonicalization
• Large-Scale Datasets
• Multi-View Geometry
• Real-World Objects
• Foundation Models

Code references

• GenIntel/Every9D

Best for: Research Scientist, AI Scientist, Computer Vision Engineer, Robotics Engineer

Related on AIssential

• Towards Learning a Generalizable 3D Scene Representation from 2D Observations — A Generalizable Neural Radiance Field predicts 3D workspace occupancy from robot observations in a global frame.
• 🍒Count Anything, Any Granularity🍒 👉Open-world counting as multi-grained counting, where... — CAAG enables flexible, multi-grained object counting using visual exemplars and fine-grained text specifications.
• VideoNet: A Large-Scale Dataset for Domain-Specific Action Recognition — VideoNet introduces a large-scale dataset and benchmark to re-evaluate VLM performance on domain-specific action recognition.
• Exploring 6D Object Pose Estimation with Deformation — DeSOPE dataset enables 6D pose estimation for deformed objects, revealing existing methods struggle with non-rigid shapes.
• LongStream: Long-Sequence Streaming Autoregressive Visual Geometry — LongStream enables stable, metric-scale 3D reconstruction for ultra-long sequences by decoupling gauge and optimizing Transformer caches.
• 🪝Syn4D: Multiview Synthetic 4D Dataset🪝 👉Syn4D is novel multi-view synthetic dataset of... — Syn4D is a synthetic multi-view 4D dataset offering comprehensive ground-truth for dynamic scene analysis.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.