MinShap: A Modified Shapley Value Approach for Feature Selection

2026-04-17 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

MinShap is a novel feature selection algorithm that adapts the classic Shapley value framework to address its limitations in distinguishing direct from indirect feature effects. Developed from a directed acyclic graphical (DAG) model perspective, MinShap considers the minimum marginal contribution across feature permutations, rather than the average, to identify features with direct influence on the target variable. The algorithm provides theoretical guarantees for Type I error control and is connected to multiple hypothesis testing and p-value approaches, enhancing performance in lower-sample and higher-noise settings. Numerical simulations across linear, non-linear, conditional interaction, and logistic models, using XGBoost and neural networks, demonstrate that MinShap and its related Max-p algorithm consistently outperform state-of-the-art methods like LOCO, GCM, and Lasso in terms of accuracy, F1 score, and stability, while maintaining Type I error control. Real-world applications to wine quality and California housing datasets further validate its superior performance and stability.

Key takeaway

For AI Engineers and Research Scientists tasked with robust feature selection in complex, non-linear models with highly dependent features, MinShap offers a statistically sound and empirically superior alternative to traditional methods. Its ability to distinguish direct from indirect feature effects, coupled with strong performance in accuracy and stability, means you can build more interpretable and reliable models. Consider integrating MinShap, especially when dealing with sparse datasets or when existing methods like LOCO or GCM yield unstable or inaccurate results.

Key insights

MinShap adapts Shapley values for robust feature selection by focusing on minimum marginal contributions across permutations.

Principles

• Faithfulness assumption links conditional independence to DAG structure.
• Minimum marginal contribution identifies direct feature effects.
• Multiple hypothesis testing improves power in finite-sample regimes.

Method

MinShap replaces the average marginal contribution in Shapley values with the minimum contribution across permutations, leveraging DAG faithfulness. It uses a threshold derived from variance estimates for Type I error control and can be extended with adjusted p-value methods.

In practice

• Use MinShap for feature selection in non-linear models with dependent features.
• Consider adjusted p-value methods for sparse models or small sample sizes.
• Parallelize computation of marginal contributions for efficiency.

Topics

• MinShap Algorithm
• Feature Selection
• Shapley Values
• Directed Acyclic Graphical Models
• Multiple Hypothesis Testing

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Data Scientist

Related on AIssential

• MinShap: A Modified Shapley Value Approach for Feature Selection — MinShap adapts Shapley values for feature selection by focusing on minimum marginal contributions, improving accuracy and stability.
• Statistical Inference and Learning for Shapley Additive Explanations (SHAP) — New methods enable statistical inference and confidence intervals for global SHAP values, addressing a critical gap in explainable AI.
• Optimal structure learning and conditional independence testing — Optimal structure learning rates for poly-forests are directly determined by minimax optimal conditional independence testing rates.
• Unsupervised feature selection using Bayesian Tucker decomposition — Bayesian Tucker decomposition (BTuD) offers a promising unsupervised feature selection method for diverse datasets.
• Mamba-SSM with LLM Reasoning for Feature Selection: Faithfulness-Aware Biomarker Discovery — LLM reasoning significantly improves biomarker discovery performance by precisely removing confounders, even with imperfect biological recall.
• MOELIGA: a multi-objective evolutionary approach for feature selection with local improvement — MOELIGA refines feature selection using multi-objective genetic algorithms with local improvement and diverse fitness sharing.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.