Bounding Boxes as Goals: Language-Conditioned Grasping via Neuro-Symbolic Planning

2026-06-11 · Source: Artificial Intelligence · Field: Technology & Digital — Robotics & Autonomous Systems, Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

GRASP (Grounded Reasoning and Symbolic Planning) is a new framework addressing open-vocabulary tabletop manipulation, enabling robots to adapt to natural-language prompts in real time. It overcomes limitations of current Vision-Language Model (VLM) approaches that are often computationally intensive or require extensive training. GRASP translates natural-language queries into neuro-symbolic goal states, grounding them in the physical world using a bounding-box detection pipeline. This allows robots to interpret abstract spatial concepts like "top shelf" and execute tasks without additional fine-tuning or task-specific training. The framework demonstrated a 73.3% overall success rate across 90 real-robot trials, tested at three distinct difficulty levels.

Key takeaway

For robotics engineers developing systems for household or industrial environments, GRASP offers a significant advancement in real-time, language-conditioned manipulation. You can now implement open-vocabulary tabletop tasks without extensive fine-tuning or thousands of demonstrations, interpreting abstract spatial concepts directly. This approach reduces development time and computational overhead, allowing for more adaptable and deployable robotic solutions.

Key insights

GRASP enables robots to interpret natural language and perform open-vocabulary grasping using VLMs and bounding boxes, achieving 73.3% success without fine-tuning.

Principles

• VLMs enable zero-shot generalization in TAMP.
• Neuro-symbolic planning grounds language in physical world.
• Bounding-box detection facilitates real-world grounding.

Method

GRASP uses a pretrained VLM to translate natural-language queries into neuro-symbolic goal states, which are then grounded via a bounding-box detection pipeline for task execution.

In practice

• Interpret "top shelf" for object placement.
• Execute tasks without task-specific training.
• Adapt to diverse natural-language prompts.

Topics

• Robotics
• Vision-Language Models
• Neuro-Symbolic Planning
• Open-Vocabulary Manipulation
• Bounding Box Detection
• Zero-Shot Generalization

Best for: Research Scientist, Robotics Engineer, AI Scientist

Related on AIssential

• Bounding Boxes as Goals: Language-Conditioned Grasping via Neuro-Symbolic Planning — GRASP combines VLMs and symbolic planning for zero-shot, language-conditioned robotic manipulation using bounding box goals and closed-loop control.
• SECOND-Grasp: Semantic Contact-guided Dexterous Grasping — Integrating semantic reasoning with physical stability enhances robotic dexterous grasping performance and adaptability.
• Bridging the Morphology Gap: Adapting VLA Models to Dexterous Manipulation via Intent-Conditioned Fine-Tuning — Adapting VLA models to dexterous hands is possible by decoupling control via a virtual grasp intent and a two-stage learning architecture.
• CLASP: Language-Driven Robot Skill Selection and Composition using Task-Parameterized Learning — CLASP integrates TP-KMPs with VLMs for data-efficient, language-driven robot skill learning, selection, and composition.
• GroundedPlanBench: Spatially grounded long-horizon task planning for robot manipulation — Jointly planning actions and spatial grounding improves robot task success and action accuracy for VLMs.
• Chameleon: Control-Indexed Prospective Memory for Visuomotor Manipulation — Bio-inspired episodic memory with geometry-grounded encoding and goal-directed recall improves robotic manipulation under perceptual aliasing.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.