Auto-Fill: Learning to Predict Missing Values Accurately with Specialist Language Models

2026-07-22 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

The Auto-Fill approach addresses the challenges of costly and overconfident missing-value prediction in tabular data, a fundamental problem in data cleaning. It proposes post-training three specialist small language models (SLMs), each optimized for world knowledge, text-based reasoning, or code-based reasoning. A calibrated ensemble mechanism dynamically selects the most confident specialist or abstains, ensuring high accuracy. Extensive experiments across 11 benchmarks and 2200 real tables demonstrate Auto-Fill's superior accuracy compared to state-of-the-art reasoning models like o3-pro, Gemini 3 Pro, and DeepSeek R1. Crucially, Auto-Fill operates at less than 1% of the cost of these frontier models, highlighting the effectiveness of specialization and calibrated abstention in tabular data.

Key takeaway

For Data Scientists and Machine Learning Engineers struggling with the high cost and overconfidence of current missing-value prediction models in tabular data, Auto-Fill offers a compelling alternative. You should consider exploring this approach, which leverages specialist small language models and calibrated abstention, to achieve superior accuracy at a fraction of the cost. Investigate the publicly available Auto-Fill implementation to integrate these efficiencies into your data cleaning workflows.

Key insights

Specialized small language models with calibrated abstention significantly improve tabular data imputation accuracy and cost-efficiency.

Principles

• High-precision missing-value prediction requires world knowledge, text-based, and code-based reasoning.
• Specialization and calibrated abstention enhance accuracy in tabular data tasks.

Method

Post-train three specialist SLMs for world knowledge, text-based reasoning, and code-based reasoning. Employ a calibrated ensemble mechanism to dynamically select the most confident specialist or abstain.

In practice

• Utilize specialist SLMs for distinct reasoning capabilities.
• Implement calibrated ensemble mechanisms for improved prediction reliability.

Topics

• Missing Value Imputation
• Tabular Data
• Small Language Models
• Ensemble Learning
• Data Cleaning
• Cost Efficiency

Code references

• lyrain2001/auto-fill

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Data Scientist

Related on AIssential

• Auto-Fill: Learning to Predict Missing Values Accurately with Specialist Language Models — Auto-Fill uses specialized small language models and calibrated abstention to achieve high-precision, cost-effective missing value prediction in tabular data.
• Learning What Not to Impute: An Uncertainty-Aware Diffusion Framework for Meaningful Missingness — Diff-Joint jointly infers meaningful missingness and imputes observation-based missing values using a diffusion framework.
• Introducing TabFM: A zero-shot foundation model for tabular data — TabFM applies zero-shot in-context learning to tabular data, eliminating manual tuning and feature engineering for classification and regression.
• AugMask: Training Diffusion Models on Incomplete Tabular Data via Stochastic Augmentation and Masking — AugMask adapts diffusion models for incomplete tabular data by separating conditioning from supervision during training.
• CURED: Creating, Understanding, and Repairing Errors Demonstrator — CURED is a web demonstrator combining ML-based data cleaning and error models for tabular data, bridging theory and practice.
• Data filtering methods for training language models — Label error detection methods improve model performance, particularly on small, noisy datasets, outperforming random data removal.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.