SpaceDG: Benchmarking Spatial Intelligence under Visual Degradation

2026-05-21 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

SpaceDG is introduced as the first large-scale dataset and benchmark designed to evaluate the spatial intelligence of Multimodal Large Language Models (MLLMs) under realistic visual degradation. Recognizing that existing benchmarks overlook real-world issues like motion blur, low light, and lens distortion, SpaceDG addresses the robustness of MLLMs to imperfect visual inputs. The dataset comprises approximately 1M QA pairs from nearly 1,000 indoor scenes, generated using a physically grounded degradation synthesis engine integrated with 3D Gaussian Splatting (3DGS) rendering, simulating nine degradation types. SpaceDG-Bench, a human-verified benchmark, further provides 1,102 questions across 11 reasoning categories and 9 degradation types, yielding over 10K VQA instances. Evaluations of 25 MLLMs revealed significant impairment in spatial reasoning due to visual degradations. However, finetuning on SpaceDG markedly improved robustness, even exceeding human performance in degraded conditions without impacting clean image performance.

Key takeaway

For Machine Learning Engineers deploying MLLMs in real-world environments, you must account for visual degradation. Your current MLLMs likely suffer significant performance drops under conditions like blur or low light. Consider finetuning your models on degradation-aware datasets like SpaceDG to build robust spatial intelligence. This approach can improve performance in degraded conditions, potentially surpassing human benchmarks, without compromising accuracy on clean inputs.

Key insights

Visual degradations consistently and substantially impair MLLM spatial reasoning, but degradation-aware training can significantly improve robustness.

Principles

• Real-world visual inputs often contain degradations.
• MLLM spatial intelligence is vulnerable to visual degradation.
• Degradation-aware training enhances MLLM robustness.

Method

SpaceDG constructs a degradation-aware spatial understanding dataset by embedding a physically grounded degradation synthesis engine into 3D Gaussian Splatting (3DGS) rendering to simulate nine degradation types.

In practice

• Evaluate MLLMs with degraded visual inputs.
• Finetune MLLMs on degradation-aware datasets.
• Use 3DGS for realistic degradation synthesis.

Topics

• Multimodal Large Language Models
• Spatial Intelligence
• Visual Degradation
• 3D Gaussian Splatting
• Dataset Benchmarking
• Model Robustness

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• LongSpace: Exploring Long-Horizon Spatial Memory from Perception to Recall in Video — Long-horizon spatial memory in MLLMs requires integrating 3D structural cues and hierarchical, question-guided memory retrieval.
• UrbanWell: Benchmarking Multimodal Large Language Models for Spatio-Temporal Urban Wellbeing Analytics — UrbanWell benchmarks MLLMs for spatio-temporal urban wellbeing, revealing performance variability across diverse indicators.
• Spatial-Aware Reduction Framework: Towards Efficient and Faithful Visual State Space Models — Token reduction in visual State Space Models requires spatial awareness to prevent performance collapse.
• DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoning — Integrating infrared and RGB data via localized cross-attention significantly boosts MLLM robustness in degraded visual conditions.
• LOCUS: Local Visual Cue Search for Enhancing Fine-Grained Perception in Multimodal Large Language Models — LOCUS enhances MLLM fine-grained perception by internalizing local visual cue search via a training-time proxy task.
• Flat-Pack Bench: Evaluating Spatio-Temporal Understanding in Large Vision-Language Models through Furniture Assembly — Current LVLMs significantly struggle with fine-grained spatio-temporal reasoning and object interaction in complex video scenarios.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.