Conformal Prediction for Dyadic Regression Under Complex Missingness

2026-06-18 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Network Science & Graph Theory · Depth: Expert, extended

Summary

This paper introduces a novel framework for conformal prediction in dyadic regression problems, specifically addressing complex missingness mechanisms. It establishes super-uniformity of conformal prediction under distributional invariance conditions weaker than traditional exchangeability. A key theoretical contribution is a bijection argument that handles scenarios where the sample itself is a random subset of the index set, a setting previously unaddressed. The authors propose several conformal prediction procedures for jointly exchangeable arrays, including full conformal, split conformal, a row-column approach, and a selective conformal procedure. For missing elements, the framework establishes asymptotic validity for a graphon-weighted conformal procedure under a nonparametric graphon model, proving conditional validity for missing-not-at-random (MNAR) data. Empirical evaluations on synthetic data (e.g., n=200, alpha=0.1, rho_n=0.8) and the Cora network dataset (2708 publications, 10556 citations, 1433-dim features, 0.12 masking density) demonstrate the methods' coverage and width properties, with observed coverages of 0.9500 and 0.9488 for different splitting methods.

Key takeaway

For AI Scientists and Research Scientists working with network data, this framework offers robust methods for uncertainty quantification in dyadic regression, even with complex missingness or non-exchangeable data structures. You should consider applying the proposed full, split, or row-column conformal methods for sampled elements, or the graphon-weighted approach for missing entries, to ensure reliable prediction intervals. This is particularly relevant when dealing with adaptively chosen test points or data where missingness depends on unobserved variables.

Key insights

Conformal prediction can provide valid uncertainty quantification for dyadic regression with complex missingness and weaker invariance assumptions.

Principles

• Super-uniformity holds under transitive group actions, not just exchangeability.
• Joint exchangeability enables valid inference for missing-not-at-random (MNAR) data.
• Row-column approaches can yield narrower prediction intervals for symmetric arrays.

Method

Proposes full, split, row-column, and selective conformal prediction for sampled elements. For missing elements, uses graphon-weighted conformal prediction with estimated missingness probabilities.

In practice

• Use split conformal prediction for computational efficiency.
• Row-column approach improves interval width for symmetric arrays.
• Graphon-weighted conformal prediction handles MNAR data asymptotically.

Topics

• Conformal Prediction
• Dyadic Regression
• Missing Data
• Joint Exchangeability
• Network Data Analysis
• Uncertainty Quantification
• Graphon Models

Best for: AI Scientist, Research Scientist

Related on AIssential

• Learning What Not to Impute: An Uncertainty-Aware Diffusion Framework for Meaningful Missingness — Diff-Joint jointly infers meaningful missingness and imputes observation-based missing values using a diffusion framework.
• Online Conformal Prediction: Enforcing monotonicity via Online Optimization — New online conformal prediction methods ensure nested prediction sets across multiple confidence levels.
• An interpretable and trustworthy AI framework for large-scale longitudinal structure-pain association studies using data from the Osteoarthritis Initiative (OAI) — Combining uncertainty-aware deep learning with longitudinal statistical modeling improves osteoarthritis structure-pain association studies.
• Bounded Difference Concentration for Infinitely Exchangeable Sequences with Applications to AI Benchmark Uncertainty — Bounded-difference concentration for infinitely exchangeable sequences decomposes fluctuations, with mixture terms canceling for zero-sum contrasts.
• Partial Causal Structure Learning for Valid Selective Conformal Inference under Interventions — Selective conformal prediction under interventions can be robustly maintained by learning partial causal structures.
• Spectral Adaptive Conformal Prediction for Structured Non-Exchangeable Data — Spectral adaptive conformal prediction extends finite-sample coverage to non-exchangeable time series via local spectral similarity and online miscoverage level updates.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.