CrossView Suite: Harnessing Cross-view Spatial Intelligence of MLLMs with Dataset, Model and Benchmark

2026-05-18 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

The CrossView Suite addresses critical limitations in multimodal large language models' (MLLMs) cross-view spatial intelligence, which requires consistent reasoning across multiple viewpoints. Current MLLMs struggle due to a scarcity of large-scale, well-annotated training data, a lack of comprehensive benchmarks, and the absence of explicit object-level consistency mechanisms across views. The suite introduces CrossViewSet, a dataset of 1.6M samples across 17 fine-grained task types, curated using a multi-agent data engine. It also provides CrossViewBench, a scene-disjoint benchmark for systematic evaluation of cross-view spatial understanding. Finally, CrossViewer is proposed as a three-stage framework (Perception -> Alignment -> Reasoning) that uses an adaptive spatial region tokenizer to capture fine-grained object representations, explicitly aligns multi-view objects, and fuses aligned features to enhance cross-view inference.

Key takeaway

For research scientists developing MLLMs for real-world spatial intelligence, you should integrate explicit cross-view alignment mechanisms into your models. The CrossView Suite provides essential resources like the CrossViewSet dataset and CrossViewBench benchmark, which are crucial for both training and rigorously evaluating models beyond single-view perception. Consider adopting the Perception -> Alignment -> Reasoning paradigm to improve your MLLMs' ability to reason consistently across multiple viewpoints.

Key insights

Advancing MLLM spatial intelligence requires large-scale data, systematic evaluation, and explicit cross-view object alignment.

Principles

• Multi-view object consistency is critical.
• Data, benchmark, and model are interdependent.

Method

CrossViewer employs a three-stage Perception -> Alignment -> Reasoning paradigm, utilizing an adaptive spatial region tokenizer for object representation, explicit multi-view object alignment, and fused features for enhanced cross-view inference.

In practice

• Use CrossViewSet for MLLM training.
• Evaluate MLLMs with CrossViewBench.
• Implement explicit object alignment.

Topics

• Multimodal Large Language Models
• Cross-view Spatial Reasoning
• CrossView Suite
• CrossViewSet Dataset
• CrossViewBench Benchmark

Code references

• Thinkirin/Crossview-Suite

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Reason, Then Re-reason: Cross-view Revisiting Improves Spatial Reasoning — Revisiting spatial hypotheses with synthesized complementary views significantly enhances MLLM reasoning from egocentric videos.
• WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark — WorldBench offers a new, visually diverse benchmark to rigorously evaluate multimodal reasoning in AI models.
• How Multimodal Large Language Models See, Think, and Reason — MLLMs integrate visual perception into transformers by converting image patches into tokens for sequential processing and reasoning.
• ROSE: Benchmarking the Perception-to-Action Gap in Multimodal Models — MLLMs struggle to translate visual perception into context-specific actions, revealing a significant "perception-to-action gap."
• EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos — Synchronized egocentric and exocentric video views provide complementary memory cues for robust spatial-temporal reasoning.
• MLLMs Get It Right, Then Get It Wrong: Tracing and Correcting Late-Layer Textual Bias — MLLMs often correctly process visual information in early layers, but a "late-layer textual override" bias shifts final predictions towards text.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.