HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning

2026-05-14 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

HiSem, a novel hierarchical semantic disentangling network, addresses limitations in Remote Sensing Image Change Captioning (RSICC) by explicitly disentangling semantic representations of varying granularities. Existing RSICC methods often process changed and unchanged image pairs, which possess different semantic granularities, under a unified modeling strategy, leading to semantic entanglement. HiSem introduces the Bidirectional Differential Attention Modulation (BDAM) module to enhance cross-temporal interactions and amplify true change signals. Furthermore, it incorporates a Hierarchical Adaptive Semantic Disentanglement (HASD) module, which employs a coarse-grained image-level routing mechanism to distinguish changed and unchanged pairs, and a fine-grained token-level Mixture-of-Experts (MoE) block to model diverse change semantics. Experiments on two benchmark datasets show HiSem outperforms previous methods, achieving a +7.52% BLEU-4 improvement on the WHU-CDC dataset.

Key takeaway

For research scientists developing RSICC models, you should consider adopting hierarchical semantic disentangling strategies. Explicitly separating coarse-grained change judgment from fine-grained semantic understanding, as demonstrated by HiSem's +7.52% BLEU-4 improvement on WHU-CDC, can significantly enhance model accuracy and provide a more structured approach to bi-temporal scene analysis.

Key insights

Disentangling semantic granularities hierarchically improves remote sensing image change captioning performance.

Principles

• Align model design with intrinsic semantic heterogeneity.
• Amplify true change signals via discrepancy-aware attention.

Method

HiSem uses Bidirectional Differential Attention Modulation (BDAM) for cross-temporal interaction, followed by Hierarchical Adaptive Semantic Disentanglement (HASD) with image-level routing and a token-level Mixture-of-Experts (MoE) for semantic modeling.

In practice

• Use BDAM for enhancing change detection.
• Implement MoE for diverse semantic modeling.

Topics

• Remote Sensing Image Change Captioning
• Hierarchical Semantic Disentangling
• Bidirectional Differential Attention Modulation
• Hierarchical Adaptive Semantic Disentanglement
• Mixture-of-Experts

Code references

• Man-Wang-star/HiSem

Best for: Research Scientist, AI Scientist, Computer Vision Engineer, Machine Learning Engineer

Related on AIssential

• Seeking Consensus: Geometric-Semantic On-the-Fly Recalibration for Open-Vocabulary Remote Sensing Semantic Segmentation — Dynamic, on-the-fly recalibration improves remote sensing OVSS by addressing scene-specific geometric and semantic challenges.
• Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models — CEDAR disentangles vision-language model embeddings post-hoc, revealing compositional structure without increasing dimensionality, improving interpretability.
• RSRCC: A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of-N Ranking — RSRCC is a new benchmark for fine-grained, localized remote sensing change question-answering.
• Improved Knowledge Distillation for Land-Use Image Classification — Combining hard and soft supervision with KL divergence and Cosine Similarity improves knowledge distillation for land-use classification.
• STiTch: Semantic Transition and Transportation in Collaboration for Training-Free Zero-Shot Composed Image Retrieval — Training-free zero-shot CIR improves by refining LLM-generated captions and using set-to-set alignment for diverse compositions.
• AnyBand-Diff: A Unified Remote Sensing Image Generation and Band Repair Framework with Spectral Priors — AnyBand-Diff generates physically consistent remote sensing images by integrating spectral priors and multi-scale physical constraints into a diffusion model.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.