A multi-task learning approach combining regression and classification tasks for joint feature selection

2026-04-17 · Source: Machine learning : nature.com subject feeds · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

A new multi-task learning (MTL) method, MTLComb, addresses the challenge of combining mixed regression and classification tasks for unbiased joint feature selection, particularly in high-dimensional biomedical data. Traditional MTL frameworks struggle with heterogeneous task types due to differing loss function scales, leading to biased feature identification. MTLComb introduces an analytical loss-balancing mechanism that aligns the regularization paths of these diverse loss functions, ensuring consistent feature selection across tasks using a shared sparsity-inducing regularization. The method's efficiency and clinical utility were demonstrated through simulations and real-world biomedical studies on sepsis and schizophrenia, showing superior prediction performance, increased model stability, and improved biological interpretability compared to conventional approaches. The code for MTLComb is publicly available on GitHub.

Key takeaway

For AI Scientists and Machine Learning Engineers working with high-dimensional data involving both regression and classification tasks, MTLComb offers a robust solution for unbiased joint feature selection. Its ability to analytically balance heterogeneous loss functions and align regularization paths can lead to more stable, interpretable, and reproducible feature sets. You should consider integrating MTLComb into your workflow, especially for biomedical applications like biomarker discovery, to improve model performance and gain deeper mechanistic insights.

Key insights

MTLComb balances mixed regression and classification tasks for unbiased joint feature selection by aligning regularization paths.

Principles

• Loss function magnitudes cause regularization path misalignment in mixed-task MTL.
• Analytically rescaling task-specific losses can harmonize regularization paths.
• Joint feature selection benefits from shared sparsity-inducing regularization.

Method

MTLComb employs a principled closed-form weighting scheme for logistic and least-squares losses, aligning their regularization paths. It uses an accelerated proximal gradient descent method for optimization and estimates the regularization path by calculating a consistent $\lambda_{\text{max}}$ for mixed losses.

In practice

• Apply MTLComb for biomarker discovery in biomedical analysis.
• Use MTLComb for integrated data analysis in heterogeneous prediction tasks.
• Consider MTLComb for comorbidity analysis with shared feature spaces.

Topics

• Multi-task Learning
• Feature Selection
• Regression and Classification
• Loss Weighting Scheme
• Sepsis Prediction

Code references

• transbioZI/MTLComb

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Strategic Feature Selection — Excluding features based on manipulability alone is suboptimal; joint consideration of manipulability and predictability is key.
• Sample Selection Using Multi-Task Autoencoders in Federated Learning with Non-IID Data — Multi-task autoencoders and outlier detection enhance federated learning performance by filtering noisy, redundant, or malicious samples.
• TabPFN-MT: A Natively Multitask In-Context Learner for Tabular Data — TabPFN-MT extends Prior-Data Fitted Networks to natively support efficient multitask in-context learning for tabular data.
• Mamba-SSM with LLM Reasoning for Feature Selection: Faithfulness-Aware Biomarker Discovery — LLM reasoning significantly improves biomarker discovery performance by precisely removing confounders, even with imperfect biological recall.
• On the Difficulty of Learning a Meta-network for Training Data Selection — Meta-learning for Training-data Selection (MTS) struggles with poor gradient signals and uninformative features, but larger batches and new features improve performance.
• Elastic Net - Explained — Elastic Net combines L1 and L2 regularization for stable feature selection and grouping of correlated variables.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine learning : nature.com subject feeds.