Dataset-Origin Signatures and Shortcut Learning in Screening Mammography AI: A Cross-Dataset Case Study

2026-07-16 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition, AI in Medical Imaging · Depth: Expert, quick

Summary

A study on AI for screening mammography found that supplementing training data with biopsy-confirmed cases from abnormal-enriched external datasets significantly reduced performance. Using an EfficientNet-B5 encoder initialized with Mammo-CLIP weights, a model trained solely on the Newfoundland and Labrador Breast Screening Dataset (NLBSD) achieved an AUC-ROC of 0.737 (95% CI [0.686, 0.785]). However, adding external positive cases from CBIS-DDSM and CMMD consistently degraded performance, yielding AUC-ROC values between 0.620 and 0.644 (DeLong test, Holm-corrected p < 0.05). This degradation worsened with more external sources. A diagnostic test revealed that datasets could be almost perfectly separated by origin despite identical preprocessing, indicating persistent dataset-specific "signatures." These findings highlight that naive pooling introduces domain shift, negating the benefits of additional positive cases and necessitating domain-aware strategies for combining heterogeneous mammography datasets.

Key takeaway

For AI scientists and machine learning engineers developing screening mammography models, you should critically evaluate data pooling strategies. Naïvely combining abnormal-enriched external datasets with screening data can introduce domain shift, significantly degrading model performance. Prioritize domain-aware data integration methods to ensure robust and reliable AI, rather than simply increasing positive case volume. Your focus should be on mitigating dataset-specific "signatures" that persist even after preprocessing.

Key insights

Naïvely pooling abnormal-enriched mammography datasets introduces domain shift, degrading AI performance despite more positive cases.

Principles

• Dataset-specific characteristics persist after normalization.
• Domain shift can outweigh benefits of increased data volume.
• Evaluate data combination strategies rigorously.

Method

The study reframed the task as predicting each examination's dataset of origin to diagnose dataset-specific characteristics influencing learned representations.

In practice

• Test for dataset-origin signatures in combined data.
• Implement domain-aware strategies for heterogeneous datasets.
• Prioritize data quality over raw quantity for screening AI.

Topics

• Screening Mammography AI
• Dataset Domain Shift
• EfficientNet-B5
• Mammo-CLIP
• Medical Imaging Datasets
• Shortcut Learning

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Token-Based Dual-view Fusion and Adaptation of Large Vision Models for Breast Cancer Classification — A token-centric, multi-depth fusion framework enhances breast cancer classification by integrating complementary mammography views.
• Domain Shift in Echocardiography: Interpretable Quantification and Prediction of Cross-Dataset Left Ventricular Segmentation — Echocardiographic domain shift is measurable and its impact on LV segmentation can be predicted and mitigated.
• Unsupervised Domain Adaptation for Calcification Classification in Mammography Across Multi-Site Datasets — Unsupervised style transfer effectively mitigates domain shifts in mammography, improving calcification classification generalization across diverse datasets.
• Improving breast cancer screening workflows with machine learning — Google Research, in partnership with NHS organizations, conducted the Artificial Intelligence in Mammography Screening (AIMS) study, publishing two companion studies in Nature Cancer to evaluate an AI…
• BreastGPT: A Multimodal Large Language Model for the Full Spectrum of Breast Cancer Clinical Routine — Workflow-aligned data and cross-scale visual modeling are critical for clinically grounded medical MLLMs.
• Towards Global AI-Driven Cervical Cancer Screening — A multi-task deep learning model validated on diverse global data outperforms experts in cervical cancer screening.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.