OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Data Science & Analytics · Depth: Expert, extended

Summary

OpenGround is a novel zero-shot framework for open-world 3D visual grounding, addressing the limitation of existing methods that rely on pre-defined Object Lookup Tables (OLTs). Developed by Nanjing University and China Mobile Zijin Innovation Institute, OpenGround introduces the Active Cognition-based Reasoning (ACR) module. This module progressively augments Visual Language Model (VLM) cognition by performing human-like perception via a cognitive task chain and dynamically updating the OLT with contextually relevant objects. The framework supports both pre-defined and open-world categories. A new dataset, OpenTarget, with over 7,000 object-description pairs, was created for evaluation. Experiments show OpenGround achieves competitive performance on Nr3D, state-of-the-art on ScanRefer, and a substantial 17.6% improvement on OpenTarget, demonstrating its superior capability in open-world scenarios.

Key takeaway

For robotics engineers developing intelligent environmental perception systems, OpenGround offers a critical advancement for handling unforeseen objects. Your systems can now dynamically adapt to novel items not in pre-defined object lookup tables, significantly improving robustness in cluttered or newly reconstructed environments. Consider implementing a cognitive task chain and active perception to enhance your 3D visual grounding capabilities, moving beyond static object recognition to truly open-world understanding.

Key insights

Dynamic OLT expansion via active cognition enables 3D visual grounding for novel, undefined objects in open-world scenes.

Principles

• Human-like cognitive planning improves grounding accuracy.
• Progressive contextual perception enhances VLM understanding.
• Dynamically updated OLTs overcome static object limitations.

Method

The ACR module constructs a cognitive task chain, then uses Active Cognition Enhancement (ACE) to perceive novel objects via 2D segmentation and 3D lifting, extending the OLT for single-step grounding.

In practice

• Decompose complex queries into sequential grounding sub-tasks.
• Actively perceive objects around previously grounded references.
• Integrate 2D segmentation models for 3D object discovery.

Topics

• 3D Visual Grounding
• Open-World Perception
• Vision-Language Models
• Object Lookup Table
• Active Cognition
• Robotics Perception
• OpenTarget Dataset

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• Grounded 3D-Aware Spatial Vision-Language Modeling — GR3D integrates explicit 2D, implicit 2D, and monocular 3D grounding to enhance spatial understanding in vision-language models.
• Active Zero: Self-Evolving Vision-Language Models through Active Environment Exploration — Active environment exploration with co-evolving agents significantly improves VLM reasoning and general understanding.
• A Modular Vision-Language-Action Robotics Framework for Indoor Environments — The framework integrates perception and language processing to enable robots to act on natural language commands in indoor environments.
• Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds — Active visual attention in 3D environments requires pre-planning observations before executing camera motion.
• Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMM — Ground3D-LMM unifies 3D point grounding and metric spatial reasoning within an interactive conversational framework.
• ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop — Embodied spatial intelligence requires active perception-action loops, where agents actively seek information rather than passively processing it.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.