From Boundaries to Semantics: Prompt-Guided Multi-Task Learning for Petrographic Thin-section Segmentation

2026-04-16 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

Petro-SAM is a novel two-stage, multi-task framework designed for high-quality joint grain-edge segmentation (GES) and lithology semantic segmentation (LSS) on petrographic thin-section images. These two tasks are crucial for quantifying rock fabric and composition but traditionally suffer from separate treatment and high annotation costs. Petro-SAM addresses challenges like severe domain gaps due to extinction-dependent color variations and ultra-fine grain boundaries, which hinder direct adaptation of foundation models like the Segment Anything Model (SAM). The framework integrates seven polarized views using a Merge Block to resolve extinction issues and employs multi-scale feature fusion and color-entropy priors to refine segmentation detection, building upon SAM's robust boundary alignment capabilities.

Key takeaway

For Computer Vision Engineers developing geological imaging solutions, Petro-SAM offers a robust approach to joint grain-edge and lithology segmentation. You should consider its multi-view integration and prior-guided refinement techniques to overcome domain gaps and improve accuracy in petrographic image analysis, potentially reducing reliance on extensive expert-annotated datasets.

Key insights

Petro-SAM enables high-quality joint grain-edge and lithology segmentation in petrographic images by adapting SAM.

Principles

• Integrate multi-view data to overcome domain-specific challenges.
• Refine segmentation with multi-scale features and domain priors.

Method

Petro-SAM uses a two-stage, multi-task framework based on SAM, incorporating a Merge Block for multi-view integration and multi-scale feature fusion with color-entropy priors for refinement.

In practice

• Apply multi-view integration for challenging image domains.
• Use color-entropy priors to enhance fine-grain detection.

Topics

• Petrographic Thin-section Segmentation
• Grain-edge Segmentation
• Lithology Semantic Segmentation
• Segment Anything Model
• Multi-Task Learning

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Domain-Guided Prompting of the Segment Anything Model for Seismic Interpretation: The Role of Attributes, Visualization, and Hybrid Prompts — Zero-shot SAM adaptation for seismic interpretation is achieved by combining domain-guided visualization with hybrid prompting.
• SAM3 Self-Distillation for Fine-Grained GOOSE 2D Semantic Segmentation — Self-distillation using SAM3 as a teacher, combined with image-level multi-scale augmentation and aggressive photometric distortion, significantly boosts fine-grained semantic segmentation.
• Plan2Map: A Multimodal Benchmark for Document-Grounded Geospatial Boundary Reconstruction from Planning Records — Plan2Map enables multimodal geospatial boundary reconstruction from diverse planning documents via structured processing.
• Amodal SAM: A Unified Amodal Segmentation Framework with Generalization — Amodal SAM extends SAM's generalization to predict complete object shapes, including occluded regions, using synthetic data and novel learning.
• Robust Multispectral Semantic Segmentation under Missing or Full Modalities via Structured Latent Projection — Structured latent projection in CBC-SLP preserves modality-specific information, enhancing multimodal semantic segmentation robustness.
• Colour Extraction Pipeline for Odonates using Computer Vision — A computer vision pipeline segments Odonates and extracts color data to enable large-scale ecological correlation analysis.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.