Augmenting Human Evaluation with LLM Judges: How Many Human Reviews Do You Need?

2026-05-19 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

This paper introduces a novel framework for augmenting human evaluation of AI systems, particularly Large Language Models (LLMs), by integrating LLM-generated ratings as auxiliary data rather than as a direct substitute for human judgment. The proposed method reframes LLM evaluation as a two-stage sampling design, where LLM ratings are collected for all observations in the first stage, and human ratings are obtained for a subsample in the second stage. Utilizing a doubly robust estimator from missing data literature, the framework provides a formal basis for study design, including sample size calculations for both human and LLM ratings to achieve a targeted statistical power. It also demonstrates how to efficiently allocate human ratings, especially for evaluation types where LLM predictability is lower, thereby optimizing resource use and enhancing the rigor of AI system validation in high-stakes applications like clinical quality monitoring and regulatory assessment.

Key takeaway

For AI scientists and MLOps engineers designing evaluation studies for LLMs in high-stakes domains, you should adopt a two-stage sampling approach that integrates LLM ratings as auxiliary data. This method allows you to formally calculate the optimal number of human and LLM reviews needed to achieve desired statistical precision, ensuring rigorous validation while managing annotation costs. Prioritize human review allocation to areas where LLM predictive quality is lower to maximize efficiency and maintain robust oversight.

Key insights

Augment human evaluation with LLM ratings using a two-stage sampling design and a doubly robust estimator.

Principles

• Human ratings are the gold standard.
• LLM ratings serve as auxiliary data.
• Validity of inference is guaranteed by design.

Method

Employ a two-stage sampling design: collect LLM ratings for all units, then human ratings for a subsample. Use a doubly robust estimator to handle incomplete human data, leveraging known response probabilities.

In practice

• Determine human/LLM sample sizes for target power.
• Allocate more human reviews where LLM predictability is low.
• Use pilot data to estimate R-squared for sample size planning.

Topics

• LLM Evaluation
• Two-stage Sampling Design
• Doubly Robust Estimator
• Sample Size Calculation
• Human Oversight

Best for: AI Scientist, MLOps Engineer, Research Scientist

Related on AIssential

• Metric Match: A Subset Selection Approach to Evaluating LLM Judge Reliability — Metric Match efficiently evaluates LLM judge reliability by strategically selecting human annotation subsets, significantly reducing costs and annotation needs.
• LLMs Can Better Capture Human Judgments--With the Right Prompts — Simple prompting strategies, including eliciting statistical distributions and ensuring scenario clarity, significantly improve LLM alignment with human judgment.
• AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot — AI-assisted peer review is feasible at scale, offering benefits in accuracy and thoroughness over human reviews.
• PRAIB: Peer Review AI Benchmark of Behaviour of LLM-Assisted Reviewing — LLMs diverge significantly from human peer review behavior, necessitating specialized benchmarks for reliable integration.
• Allocating Human Oversight in AI-Enabled Analytics — An adaptive UCB algorithm efficiently allocates human verification budget in LLM-augmented tasks by learning real-time reliability.
• How strongly do you believe LLM judges on the for the ML papers?? [D] — LLM judges are useful for initial ML paper review checks but require human validation for nuanced contributions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.