Auditing Construct Overlap in Explainable Machine Learning: Evidence from Burnout-Depression Prediction Across Student Cohorts

2026-07-12 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research, Mental Health & Psychological Support · Depth: Expert, quick

Summary

An audit of Explainable Machine Learning (XML) pipelines reveals that apparently robust, cross-population-stable risk hierarchies in composite mental health outcome prediction can be artifacts of outcome construction. Researchers applied an ElasticNet pipeline to 886 medical students at the University of Lausanne (2022), validating it across 2,580 longitudinal observations and 701 non-medical students. The pipeline consistently identified trait anxiety and health satisfaction as dominant factors, achieving Kendall $τ= 1.0$ for the top-two positions across five evaluation sets and transfer performance ($R^2$) between 0.41-0.49. However, residualization experiments showed that when trait anxiety (STAI-T) was residualized against the depression subscale (CES-D, $r = 0.72$), the model's $R^2$ dropped from 0.41 to 0.16, and STAI-T's rank fell from 1 to 6. Residualizing burnout subscales against CES-D further collapsed $R^2$ to 0.016. Prediction intervals averaged 35.4 units, precluding individual-level deployment. The study's transferable contribution is a residualization protocol for XAI studies.

Key takeaway

For AI Scientists and Research Scientists interpreting Explainable Machine Learning (XAI) results in mental health prediction, you must audit for construct overlap. Your models' apparent stability and feature importance can be artifacts of how outcomes are constructed, as demonstrated by $R^2$ drops from 0.41 to 0.016 after residualization. Implement the proposed residualization protocol to validate feature importance before drawing conclusions or considering individual-level deployment, especially given average prediction intervals of 35.4 units.

Key insights

XAI pipelines can produce misleadingly stable risk hierarchies due to construct overlap in mental health outcomes.

Principles

• Correlated constructs can create artifactual stability.
• High $R^2$ does not guarantee valid feature importance.
• Individual-level XAI deployment requires narrow prediction intervals.

Method

Apply a residualization protocol: isolate shared variance between correlated predictor and outcome constructs via regression before interpreting XAI stability.

In practice

• Check for construct overlap in XAI studies.
• Use residualization to validate feature importance.
• Avoid individual deployment if prediction intervals are wide.

Topics

• Explainable Machine Learning
• Construct Overlap
• Mental Health Prediction
• ElasticNet
• Residualization Protocol
• Feature Importance

Best for: AI Scientist, Research Scientist, Data Scientist

Related on AIssential

• Reading Task Failure Off the Activations: A Sparse-Feature Audit of GPT-2 Small on Indirect Object Identification — A sparse-feature audit of GPT-2 small reveals task failure correlates, distinguishing causation from interpretability gains.
• Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models — The framework unifies XAI evaluation by scoring fidelity, simplicity, and stability to build a knowledge base for transparent, trustworthy AI.
• Towards Isolated Interventions via Almost Orthogonal Features in Language Models — Orthogonality regularization of language model features enables isolated causal interventions while preserving performance.
• Quantifying Explanation Consistency: The C-Score Metric for CAM-Based Explainability in Medical Image Classification — The C-Score quantifies explanation consistency in CAM methods, revealing model instability beyond classification metrics.
• Explainable AI based ensemble model for the identification of Schizophrenia prodromal phase — An XAI-integrated ensemble ML model accurately predicts Schizophrenia prodromal symptoms for early clinical intervention.
• Unsupervised Evaluation of Explanations for Hate Speech Classification in Portuguese — An unsupervised framework evaluates XAI explanation faithfulness by measuring model performance degradation upon keyword removal.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.