SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents

2026-07-16 · Source: Artificial Intelligence · Field: Technology & Digital — Robotics & Autonomous Systems, Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

SafeRelBench is a new spatial-relation-aware safety benchmark designed to evaluate process-level safety in VLM-driven embodied agents. It addresses a gap in existing evaluations, which often overlook how actions change physical scenes over time, focusing instead on static risk or final task completion. The benchmark comprises 507 executable evaluation samples, including 248 spatial-relation samples and 259 non-spatial control samples. Evaluations of seven open- and closed-source VLM-driven agents using SafeRelBench revealed a significant disparity: models frequently achieve task success while failing to comply with process-level safety constraints. This highlights that current embodied intelligence lacks reliable reasoning about how object relations influence risk during interaction.

Key takeaway

For robotics engineers developing VLM-driven embodied agents, you must prioritize process-level safety beyond mere task completion. Your current models likely complete tasks but violate safety constraints related to spatial relations. Implement explicit checks for object relations like support and containment *before* executing actions to mitigate risks and ensure safer interaction in dynamic environments. This approach will improve agent reliability and prevent unintended hazardous outcomes.

Key insights

VLM-driven embodied agents often complete tasks but fail process-level safety due to poor spatial relation reasoning.

Principles

• Process-level safety requires dynamic spatial relation reasoning.
• Task success does not guarantee safety compliance in embodied agents.

Method

SafeRelBench explicitly tests safety conditions *before* risk-prone actions, using spatial relations like support, containment, and proximity. It includes 507 executable samples, split into spatial-relation and non-spatial controls, to evaluate VLM-driven agents.

In practice

• Integrate spatial relation checks into agent planning modules.
• Prioritize pre-action safety condition verification in VLM training.

Topics

• Embodied AI
• Vision-Language Models
• Robotics Safety
• Spatial Relations
• Process-Level Safety
• Benchmarking

Best for: Research Scientist, AI Scientist, Robotics Engineer

Related on AIssential

• SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models — MLLMs recognize hazards in QA but struggle with embodied mitigation, revealing a critical safety alignment gap.
• Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic Manipulation — Safety in long-horizon robotic manipulation requires a cross-layer analysis across planning, policy, and execution to address fragmented literature and persistent gaps.
• SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction — SeerGuard proactively assesses mobile GUI agent actions for risks using a world model before execution.
• the openclaw trust layer — Real-world AI workflow failures stem from a lack of control and safety boundaries, not model weakness.
• Symbolic Guardrails for Domain-Specific Agents: Stronger Safety and Security Guarantees Without Sacrificing Utility — Symbolic guardrails offer provable safety and security for domain-specific AI agents without compromising utility.
• HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark — Intrinsic risks in long-horizon AI agents under benign conditions pose a significant, underexplored safety challenge.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.