GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

The paper introduces HIOcc, a hierarchical indoor occupancy benchmark that unifies ScanNet, ScanNet++, and Matterport3D into a common sparse semantic occupancy format. HIOcc addresses limitations of prior single-view or room-level benchmarks by supporting local semantic occupancy prediction, room-level online mapping, and building-level mapping across connected panoramic environments. Complementing this, the authors propose GEM-Occ, a Gaussian Evidence Memory framework for semantic occupancy mapping. GEM-Occ treats local visual geometry predictions as transient evidence, converting them into semantic Gaussian occupancy and free-space ray evidence. This evidence is fused into a persistent hierarchical memory, comprising local caches, room-level submaps, and a building-level graph, using visibility- and uncertainty-aware causal updates. Experiments on HIOcc demonstrate that GEM-Occ improves local occupancy prediction, online map stability, free-space reasoning, revisit consistency, and building-level scalability over existing baselines, achieving 61.37 IoU and 57.76 mIoU for local prediction.

Key takeaway

For Robotics Engineers developing embodied agents for complex indoor navigation, this work highlights the necessity of robust, scalable semantic occupancy mapping. You should consider adopting hierarchical memory structures and causal fusion techniques, like those in GEM-Occ, to maintain stable, long-horizon maps across connected spaces. Prioritizing explicit free-space modeling and uncertainty-aware updates will significantly improve map consistency and revisit performance, crucial for reliable autonomous operation in multi-room environments.

Key insights

Persistent semantic occupancy mapping for embodied agents requires hierarchical memory and causal fusion of transient visual evidence.

Principles

• Treat local geometry predictions as transient evidence.
• Explicitly model free space and unknown regions.
• Fuse evidence with visibility- and uncertainty-awareness.

Method

GEM-Occ converts local visual geometry predictions into semantic Gaussian occupancy and free-space ray evidence. This evidence is fused into a hierarchical memory via causal, confidence-weighted updates.

In practice

• Use Gaussian primitives as long-lived map elements.
• Implement temporal decay for memory updates.
• Periodically merge and prune low-confidence Gaussians.

Topics

• Semantic Occupancy Mapping
• Embodied AI
• Gaussian Splatting
• Hierarchical Memory
• Indoor Robotics
• 3D Scene Understanding
• HIOcc Benchmark

Code references

• OpenDriveLab/OpenScene

Best for: Research Scientist, Computer Vision Engineer, AI Scientist, Robotics Engineer, Machine Learning Engineer

Related on AIssential

• MatterDoor: Sampling Zero-shot Spatio-semantic Priors using Generative Models — Generative models can sample spatio-semantic priors for robotic planning in partially observed 3D environments.
• Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy — OmniAct enables persistent embodied agents through a hierarchical, asynchronous architecture for unified cyber-physical autonomy and failure recovery.
• MVP-Nav: Multi-layer Value Map Planner Navigator — MVP-Nav enables physically-aware, RGB-only zero-shot object navigation by integrating 3D foundation models and a Multi-layer Value Map.
• RADIO-ViPE: Online Tightly Coupled Multi-Modal Fusion for Open-Vocabulary Semantic SLAM in Dynamic Environments — RADIO-ViPE enables robust, calibration-free, open-vocabulary semantic SLAM in dynamic environments using monocular RGB video.
• ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop — Embodied spatial intelligence requires active perception-action loops, where models currently exhibit "action blindness" and metacognitive gaps.
• Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots — Embodied.cpp unifies embodied AI model deployment across diverse hardware, addressing multi-rate, latency-first inference needs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.