VTM-Nav: Harnessing Cross-Episode Experience for Object-Goal Navigation with Hierarchical Visual-Topological Memory

2026-07-27 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

VTM-Nav is a novel training-free vision-language model (VLM) navigation framework designed for cross-episode object-goal navigation, where an embodied agent repeatedly operates in the same indoor scene. It introduces a persistent hierarchical Visual-Topological Memory (VTM) that accumulates self-acquired scene experience, organizing knowledge at both room and object levels for coarse-to-fine retrieval. This system addresses the limitation of standard episodic protocols that discard scene-specific state after each run. VTM-Nav also incorporates a conservative execution guard to mitigate navigation failures like oscillations and premature stopping. Evaluated on HM3D v0.1, HM3D v0.2, and MP3D benchmarks, VTM-Nav achieved superior performance, reaching 59.6% Success Rate (SR) on HM3D v0.1, 72.0% SR on HM3D v0.2, and 44.3% SR on MP3D, consistently outperforming WMNav* and a strengthened WMNav + Textual Memory* baseline while maintaining comparable Success weighted by Path Length (SPL). Its effectiveness stems from structured visual-topological experience reuse.

Key takeaway

For Robotics Engineers developing embodied agents for repeated indoor tasks, VTM-Nav demonstrates that persistent, structured scene memory significantly enhances object-goal navigation without retraining. You should integrate hierarchical visual-topological memory to accumulate and reuse cross-episode experience, improving success rates and path efficiency in familiar environments. Consider implementing a conservative execution guard to ensure memory guidance complements, rather than overrides, real-time observations, preventing common navigation failures and making your agents more robust.

Key insights

Structured hierarchical visual-topological memory enables training-free VLM agents to reuse cross-episode scene experience for improved object navigation.

Principles

• Persistent memory improves navigation over repeated scene visits.
• Hierarchical memory (room-to-object) is more effective than flat.
• Memory guidance should be soft and visually grounded.

Method

VTM-Nav extracts semantic evidence, localizes the agent in a hierarchical Visual-Topological Memory, retrieves target-conditioned priors, and applies soft memory-guided action selection with a conservative execution guard.

In practice

• Implement hierarchical memory for embodied agents in fixed environments.
• Use a conservative guard to prevent memory-induced navigation errors.
• Prioritize success-backed records for more reliable memory retrieval.

Topics

• Object-Goal Navigation
• Visual-Topological Memory
• Vision-Language Models
• Embodied AI
• Cross-Episode Learning
• Training-Free Navigation

Best for: Research Scientist, AI Scientist, Robotics Engineer, Machine Learning Engineer

Related on AIssential

• VTM-Nav: Hierarchical Visual-Topological Memory for Cross-Episode Object-Goal Navigation — VTM-Nav uses hierarchical visual-topological memory for robust, training-free cross-episode object-goal navigation.
• NavVerse: Benchmarking Indoor-to-Outdoor Embodied Navigation in Continuous Robot Simulation — NavVerse benchmarks indoor-to-outdoor robot navigation, revealing current agents struggle with cross-context adaptation and safety in complex environments.
• MEMORA: Embodied Action Memory from Egocentric Videos for Reasoning and Planning — MEMORA provides robots with persistent, editable, and consolidated embodied action memory for improved long-horizon planning and reasoning.
• Expanding Spatial and Temporal Context for Robotic Imitation Learning With Scene Graphs — Scene graphs provide explicit, compact memory for robots to reason over long-horizon tasks under partial observability.
• NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation — NavWAM integrates visual foresight with action and value targets for direct robot control in goal-conditioned navigation.
• GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory — Persistent semantic occupancy mapping for embodied agents requires hierarchical memory and causal fusion of transient visual evidence.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.