Seeking Consensus: Geometric-Semantic On-the-Fly Recalibration for Open-Vocabulary Remote Sensing Semantic Segmentation

2026-04-30 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Remote Sensing · Depth: Expert, extended

Summary

Seeking Consensus (SeeCo) is a novel, plug-and-play framework designed to enhance training-free open-vocabulary semantic segmentation (OVSS) models for remote sensing images. It addresses challenges like semantic ambiguity and incomplete foreground activation by recalibrating existing OVSS models on-the-fly during inference. SeeCo achieves this through dual consensus learning: Geometric Consensus Learning (GCL) ensures rotation-invariant representations via multi-view consistent observations, while Semantic Consensus Learning (SCL) dynamically recalibrates textual descriptions using a multi-modal collaborative prompting strategy to mitigate semantic bias. These consensus mechanisms are integrated via an Online Consensus Injector (OCI), which adaptively tunes model parameters. Extensive experiments across eight remote sensing OVSS benchmarks, including OpenEarthMap, LoveDA, and iSAID, demonstrate that SeeCo consistently improves segmentation performance, achieving up to 4.3% mIoU gains when integrated with models like ProxyCLIP, and notably improving performance on challenging datasets like Vaihingen by 10.2% to 11.9%.

Key takeaway

For research scientists developing open-vocabulary semantic segmentation solutions for remote sensing, SeeCo offers a robust, training-free enhancement. You should consider integrating its geometric and semantic consensus learning modules into your existing OVSS models to achieve significant performance gains, particularly in scenes with arbitrary orientations and high intra-class heterogeneity. This approach dynamically adapts to unique scene properties, improving segmentation accuracy without requiring extensive retraining or pixel-level annotations.

Key insights

Dynamic, on-the-fly recalibration improves remote sensing OVSS by addressing scene-specific geometric and semantic challenges.

Principles

• Maintain rotation-invariant representations for bird's-eye views.
• Alleviate semantic bias while keeping text encoders frozen.

Method

SeeCo uses Geometric Consensus Learning (GCL) for multi-view consistency and Semantic Consensus Learning (SCL) with multi-modal prompting for text recalibration, integrated via an Online Consensus Injector (OCI) during inference.

In practice

• Apply multi-view observations to enhance geometric robustness.
• Use LLMs for adaptive, enriched textual descriptions.
• Employ low-rank adaptation for efficient parameter tuning.

Topics

• Open-Vocabulary Semantic Segmentation
• Remote Sensing Images
• Geometric Consensus Learning
• Semantic Consensus Learning
• Test-Time Training

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning — Disentangling semantic granularities hierarchically improves remote sensing image change captioning performance.
• Supervised Classification Heads as Semantic Prototypes: Unlocking Vision-Language Alignment via Weight Recycling — Repurposing pretrained vision model classification heads as semantic prototypes enables efficient VLM alignment and data augmentation.
• Geometry-Aware Cross Modal Alignment for Light Field-LiDAR Semantic Segmentation — Integrating light field images and LiDAR point clouds significantly enhances semantic segmentation, especially for occluded objects.
• Geometric Consistency Protocol for Foundation Model Features in Multi-View Satellite Imagery — Geometric consistency is crucial for reliable feature matching in multi-view satellite imagery, decoupling semantic similarity from practical matchability.
• Semantic Smoothing via Novel View Synthesis for Robust SAR Image Classification — Semantic smoothing uses generative models to create semantically consistent views for robust classification, outperforming isotropic noise.
• Robust Multispectral Semantic Segmentation under Missing or Full Modalities via Structured Latent Projection — Structured latent projection in CBC-SLP preserves modality-specific information, enhancing multimodal semantic segmentation robustness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.