Cross-Fitting-Free Debiased Machine Learning with Multiway Dependence

2026-02-13 · Source: stat.ML updates on arXiv.org · Field: Science & Research — Mathematics & Computational Sciences, Research Methodology & Innovation, Machine Learning Theory · Depth: Expert, extended

Summary

This paper, authored by Kaicheng Chen and Harold D. Chiang, develops a novel asymptotic theory for two-step debiased machine learning (DML) estimators within generalized method of moments (GMM) models. Crucially, this theory operates without relying on cross-fitting, a common technique that can be statistically inefficient and computationally burdensome, especially with complex first-stage learners and multiway clustered dependence. The authors demonstrate that valid inference can be achieved by combining Neyman-orthogonal moment conditions with a localization-based empirical process approach, accommodating an arbitrary number of clustering dimensions. The resulting DML-GMM estimators are shown to be asymptotically linear and normal under multiway clustered dependence. A significant technical contribution includes the derivation of new global and local maximal inequalities for general classes of functions of sums of separately exchangeable arrays, which are foundational to their theoretical arguments and hold independent interest.

Key takeaway

For AI Researchers and Research Scientists working with DML-GMM in environments with multiway clustered data, this work suggests that abandoning traditional cross-fitting can lead to more efficient and less computationally intensive inference. You should consider implementing the proposed cross-fitting-free DML-GMM framework, particularly when dealing with high-dimensional or nonparametric nuisance components where sample splitting is costly. This approach offers a robust alternative for achieving asymptotically linear and normal estimators, potentially improving the precision of your parameter estimates in finite samples.

Key insights

Valid DML-GMM inference is possible without cross-fitting, even with multiway clustered data.

Principles

• Cross-fitting can be inefficient for complex learners and multiway clustered data.
• Neyman-orthogonal moment conditions enable cross-fitting-free DML.
• Localization-based empirical processes control nuisance estimation impact.

Method

The method combines Neyman-orthogonal moment conditions with a localization-based empirical process approach, leveraging novel maximal inequalities for separately exchangeable arrays to achieve valid inference without sample splitting in multiway clustered DML-GMM.

In practice

• Apply DML-GMM without cross-fitting for multiway clustered datasets.
• Utilize generalized linear models with L1-regularization for nuisance estimation.
• Consider regression trees or deep neural networks as first-stage learners.

Topics

• Debiased Machine Learning
• Multiway Clustering
• Generalized Method of Moments
• Empirical Process Theory
• Maximal Inequalities

Best for: AI Researcher, AI Scientist, Research Scientist

Related on AIssential

• Domain-Adaptive Model Merging Across Disconnected Modes — DMM merges divergent models data-free by synthesizing pseudo-data and distilling knowledge, outperforming existing methods.
• Debiased Counterfactual Generation via Flow Matching from Observations — Deconfounding flows efficiently estimate counterfactual distributions by transforming observational data, leveraging inherent structural similarities.
• XMSE-Aware Adaptive Empirical Bayes Estimation — An XMSE-aware mixed estimator adaptively blends ML and EB shrinkage, using second-order risk to mitigate kernel misspecification.
• Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing — Identifies degenerate Gaussian mixture latent variables from piecewise affine observations using sparsity regularization for disentanglement.
• Rethinking Training & Inference for Forecasting: Linking Winner-Take-All back to GMMs — A modeling-training mismatch in GMM-based trajectory forecasting causes uninformative posteriors, correctable post-hoc without retraining.
• Diffusion Flow Matching: Dimension-Improved KL Bounds and Wasserstein Guarantees — DFM's theoretical convergence bounds now show improved dimensional dependence for KL and 2-Wasserstein distances.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.