From Open Waters to Enclosed Cabins: ProteusVPR for Cross-Scene Visual Place Recognition in Maritime Perception and Cabin Inspection

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

ProteusVPR is a two-stage retrieval-refinement framework designed for Visual Place Recognition (VPR) in challenging maritime environments, specifically addressing transitions between open decks and enclosed cabins. The first stage uses any standard VPR model for initial image retrieval. The second stage, a geometric-visual estimation network, refines localization by fusing the retrieved image with two preceding frames, incorporating geometric descriptors, a local affine coordinate system, and camera azimuth encoding. This framework is evaluated on the new XHZ dataset, an 8K-panoramic ship-borne dataset with multi-floor cabin structures and strict query-database separation. Experiments show ProteusVPR consistently improves localization accuracy across multiple VPR backbones, reducing mean localization error by over 60% on average.

Key takeaway

For robotics engineers developing autonomous inspection systems for maritime vessels, ProteusVPR offers a robust solution for precise visual localization across diverse ship environments. You should consider integrating a two-stage refinement framework that leverages temporal context and geometric priors to significantly reduce localization errors, especially in repetitive indoor scenes. This approach enhances existing VPR backbones, improving accuracy by over 60% on average.

Key insights

Cross-scene VPR in maritime settings benefits from a two-stage retrieval-refinement approach using temporal and geometric context.

Principles

• Fusing temporal frames improves localization precision.
• Local affine coordinate systems enhance geometric robustness.
• Geometric descriptors provide essential spatial priors.

Method

A two-stage VPR process: first, standard image retrieval; second, a geometric-visual network fuses the retrieved image with two preceding frames, using DINOv2 features, attention, geometric descriptors, local affine coordinates, and azimuth encoding to regress precise location.

In practice

• Implement a second-stage refinement for VPR.
• Use DINOv2 as a frozen visual encoder.
• Incorporate temporal image sequences for context.

Topics

• Visual Place Recognition
• Maritime Robotics
• Autonomous Inspection
• Geometric-Visual Fusion
• XHZ Dataset
• Localization Accuracy

Code references

• erjieshizhe/pose-position

Best for: Research Scientist, AI Scientist, Robotics Engineer, Computer Vision Engineer

Related on AIssential

• Visual Place Recognition in Forests with Depth-Aware Distillation — Depth-aware distillation enhances forest Visual Place Recognition by integrating geometric cues into DINOv2, improving robustness.
• G2IA: Geometry-Guided Instance-Aware Retrieval and Refinement for Cross-Modal Place Recognition — G2IA improves cross-modal place recognition by combining geometry-guided retrieval with instance-aware refinement, overcoming modality gaps and perceptual aliasing.
• FRED: A Multi-Modal Autonomous Driving Dataset for Flooded Road Environments — FRED is the first multi-modal dataset for autonomous driving in flooded environments, enabling robust water hazard detection.
• EPRBench: A High-Quality Benchmark Dataset for Event Stream Based Visual Place Recognition — Event cameras and LLMs enhance Visual Place Recognition robustness and interpretability in challenging environments.
• Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization — OpenReLoc uses object-level representations and open-vocabulary semantics for robust indoor visual relocalization.
• Hardware- and Vision-in-the-Loop Validation of Deep Monocular Pose Estimation for Autonomous Maritime UAV Flight — A hardware-validated vision-in-the-loop framework enables safe, realistic indoor testing of maritime UAV pose estimation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.