CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

CAIRN is a novel topology-aware 3D-LLM designed for understanding complex multi-room household environments, addressing limitations of prior models focused on single-room scenes. It integrates transformer attention with scene hierarchy, enriching object tokens with room-local relational context via a graph neural network, introducing learned room tokens for abstraction, and applying a hierarchical attention mask with geometric bias. To evaluate this, the authors introduce CAIRN-MR, a new benchmark built on HM3D, featuring 673 scenes and 238K task annotations across grounding, captioning, and four question-answering tasks that span intra-room perception to cross-room reasoning. CAIRN, using Qwen3-8B as its backbone, significantly outperforms previous 3D-LLMs on all CAIRN-MR tasks, achieving gains up to +14.9 CIDEr on captioning, while maintaining competitive performance on five single-room benchmarks.

Key takeaway

For Machine Learning Engineers developing 3D scene understanding for embodied AI, you should recognize that single-room models are inadequate for complex household environments. CAIRN demonstrates that explicitly modeling hierarchical scene topology and spatial priors significantly improves cross-room reasoning. Consider integrating hierarchical scene graphs, learned room tokens, and topology-aware attention with geometric bias into your 3D-LLM architectures. Evaluate your models on multi-room benchmarks like CAIRN-MR to ensure robust performance in real-world, interconnected spaces.

Key insights

Explicitly modeling hierarchical 3D scene topology and spatial priors significantly enhances LLM understanding of complex multi-room environments.

Principles

• Align attention with scene hierarchy.
• Mediate cross-room communication via room tokens.
• Inject explicit spatial priors with geometric bias.

Method

Construct a hierarchical 3D scene graph, tokenize it into object and learned room tokens, then apply hierarchical masked attention with geometric bias to route information based on scene topology.

In practice

• Use CAIRN-MR to benchmark multi-room 3D scene understanding.
• Integrate graph tokens for local object relational context.

Topics

• 3D-LLMs
• Multi-room Scene Understanding
• Hierarchical Scene Graphs
• Topology-aware Attention
• CAIRN-MR Benchmark
• Embodied AI

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding — Video generation models implicitly learn robust 3D structural priors, which can enhance MLLM spatial reasoning.
• 3D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models — 3D-PLOT-LLM enables large language models to directly address and reason about 3D object parts through input token reorganization.
• Global-Local Monte Carlo Tree Search in Vision-Language Models for Text-to-3D Indoor Scene Generation — PRM-guided MCTS with hierarchical scene representation enables LVLMs to generate realistic 3D indoor scenes by correcting errors and balancing search.
• Seeing Together:Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models — Multi-robot egocentric spatial reasoning benefits from physics-informed MLLMs and specialized cooperative datasets.
• Reinforcing 3D Understanding in Point-VLMs via Geometric Reward Credit Assignment — Targeted reward assignment and geometric consistency verification significantly improve 3D spatial reasoning in Point-VLMs.
• EditSSC: Toward Editable Semantic Occupancy Scenes with Unconditional Diffusion Models — EditSSC repurposes 2D latent diffusion for 3D semantic scene generation, enabling training-free editing capabilities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.