Isotonic Survival Regression: Calibrated Survival Distributions from Deep Cox Models

· Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

Isotonic Survival Regression (ISR) is a novel post hoc calibration method designed to refine predicted survival probabilities from Deep Cox models, which are widely used for time-to-event data in life sciences and engineering. Deep Cox models effectively handle censoring and unstructured data like clinical text or genomic sequences but often produce poorly calibrated survival probabilities. ISR, specifically its Doubly Robust (DR-ISR) variant, addresses this by using isotonic regression to project pointwise survival estimates onto a set of valid survival functions that are monotonic in both time and risk. The method establishes theoretical guarantees, including a double-robustness property and asymptotic calibration, ensuring consistency even if either the event or censoring model is misspecified. Empirical evaluations on synthetic and real-world clinical data, including The Cancer Genome Atlas (TCGA) pathology reports and RNA-seq data, demonstrate that DR-ISR significantly improves calibration metrics like AUPIT and Integrated Brier Score (IBS) compared to existing baselines, without compromising discriminative power.

Key takeaway

For AI Scientists and Machine Learning Engineers developing survival analysis models in safety-critical domains, adopting Isotonic Survival Regression (ISR), particularly the Doubly Robust (DR-ISR) variant, is crucial. This method significantly improves the calibration of Deep Cox model predictions, making them more statistically reliable for informed decision-making in areas like medical prognostics. You should integrate DR-ISR as a post hoc calibration step to enhance the trustworthiness of your time-to-event forecasts, especially when dealing with high-dimensional, unstructured data and right-censoring.

Key insights

DR-ISR calibrates Deep Cox model survival predictions using isotonic regression, ensuring accuracy and robustness to model misspecification.

Principles

Method

DR-ISR constructs doubly robust pointwise survival pseudo-outcomes, then projects them onto valid, monotonic survival functions using a two-dimensional isotonic regression procedure efficiently implemented with Dykstra's algorithm and PAVA.

In practice

Topics

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.