Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis

2026-07-27 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

The paper proposes a visual relocalization method for challenging planetary-like terrains characterized by low texture, perceptual aliasing, harsh illumination, and sparse, weakly overlapping viewpoints. It employs 3D Gaussian Splatting (3DGS) as a differentiable scene representation, departing from classical correspondence-based pipelines. A key contribution is a novel geometry-aware training strategy that integrates photometric and geometric losses, combining multi-view stereo (MVS) and LiDAR depths for the first time. This joint optimization significantly improves the 3DGS model's fit to the underlying scene geometry, enhancing photometric and geometric consistency. Extensive experiments on the DLR S3LI Vulcano Dataset, a planetary-analog environment, validate the approach. Results show a substantial reduction in geometric reconstruction errors by over 74% and an improvement in 6-DoF pose recall from 6.25% to 43.20% under challenging conditions, underscoring the critical role of geometric consistency for robust relocalization.

Key takeaway

For robotics engineers developing autonomous systems for planetary-like or GNSS-denied environments, you should prioritize geometry-aware scene representations. Traditional photometric-only 3DGS models are insufficient for robust 6-DoF pose estimation in low-texture or aliased terrains. Integrate multi-view stereo and LiDAR depth supervision into your 3DGS training to achieve superior geometric consistency and significantly improve relocalization accuracy, reducing reconstruction errors and boosting pose recall in extreme conditions.

Key insights

Combining MVS and LiDAR depths in 3DGS training significantly improves visual relocalization accuracy in extreme environments.

Principles

• Geometric consistency is key for robust relocalization.
• Photometric-only 3DGS training is geometrically unstable.
• MVS and LiDAR supervision offer complementary benefits.

Method

A geometry-aware 3DGS training strategy combines photometric loss with MVS depth/normal alignment and a LiDAR-guided Chamfer loss for global metric consistency.

In practice

• Use MVSAnywhere for depth and normal supervision.
• Apply Chamfer loss for LiDAR point cloud alignment.
• Disable automatic pose normalization in Nerfstudio.

Topics

• Visual Relocalization
• 3D Gaussian Splatting
• Multi-View Stereo
• LiDAR Data
• Planetary Robotics
• Pose Estimation

Code references

• DLR-RM/multimodal-gsplat-relocalization

Best for: Research Scientist, Robotics Engineer, AI Scientist, Computer Vision Engineer

Related on AIssential

• Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis — Combining MVS and LiDAR depths for geometry-aware 3DGS training enables robust visual relocalization in challenging, low-texture environments.
• SwiftGS: Episodic Priors for Immediate Satellite Surface Recovery — SwiftGS meta-learns transferable priors for immediate, zero-shot 3D satellite surface reconstruction using a hybrid Gaussian-SDF representation.
• Towards Next-Generation SLAM: A Survey on 3DGS-SLAM Focusing on Performance, Robustness, and Future Directions — 3D Gaussian Splatting offers a new paradigm for high-fidelity, efficient, and robust SLAM systems.
• DGSG-Mind: Dynamic 3D Gaussian Scene Graphs for Long-Term Scene Understanding and Grounding — DGSG-Mind uses hybrid 3D Gaussian scene graphs and an embodied reasoning agent for robust, dynamic, long-term scene understanding.
• Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective — Feed-forward 3D reconstruction offers efficient, generalizable scene modeling by directly mapping 2D inputs to 3D representations.
• PanoPlane: Plane-Aware Panoramic Completion for Sparse-View Indoor 3D Gaussian Splatting — Panoramic scene completion with layout-anchored attention steering significantly improves sparse-view 3D reconstruction.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.