ADeLe: Predicting and explaining AI performance across tasks

2026-04-01 · Source: Microsoft Research · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, short

Summary

Microsoft researchers, in collaboration with Princeton University and Universitat Politècnica de València, have introduced ADeLe (AI Evaluation with Demand Levels), a new method for evaluating AI models. Published in "Nature," ADeLe moves beyond traditional aggregate benchmarks by characterizing both models and tasks using 18 core ability scores, such as reasoning and domain knowledge. This framework assigns tasks a 0-5 value based on required abilities and creates ability profiles for models by measuring performance changes with task difficulty. ADeLe predicts model performance on new tasks with approximately 88% accuracy for models like GPT-4o and Llama-3.1, identifying specific strengths and weaknesses. It also reveals that many existing benchmarks provide incomplete evaluations and helps explain performance differences as task complexity increases.

Key takeaway

For research scientists developing or deploying large language models, ADeLe offers a more rigorous evaluation framework than traditional benchmarks. You should consider using ADeLe's ability-based scoring to understand specific model strengths and weaknesses, predict performance on novel tasks with ~88% accuracy, and design more transparent and reliable AI systems. This approach helps explain why models succeed or fail, improving assessment accuracy.

Key insights

ADeLe evaluates AI models by matching task demands to model capabilities, predicting performance with high accuracy.

Principles

• Evaluate models and tasks using shared capability scores.
• Performance changes with task difficulty reveal true ability.
• Benchmarks often misrepresent isolated abilities.

Method

ADeLe scores tasks on 18 core abilities (0-5), builds model ability profiles, and predicts performance on new tasks by comparing model profiles to task demands.

In practice

• Diagnose existing benchmark limitations.
• Design more effective AI evaluation benchmarks.
• Anticipate model failures before deployment.

Topics

• ADeLe
• AI Evaluation
• Large Language Models
• Ability Profiles
• Performance Prediction

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• From Exams to Escape Rooms: How We Learned to Test AI — AI evaluation is a dynamic process, constantly adapting to measure increasingly complex model capabilities beyond narrow task performance.
• Google Gemma 2 — Gemma 2 offers class-leading performance and efficiency across three parameter sizes.
• ATLAS: All-round Testing of Long-context Abilities across Scales — Long-context LLM evaluation requires length-dependent capability profiling to reveal performance degradation and task-specific strengths.
• When large language models are reliable for judging empathic communication — LLMs reliably judge empathic communication, matching experts and outperforming crowdworkers when properly benchmarked.
• Human Evaluation of Large Language Models: How Good is Hugging Face’s BLOOM? — Human evaluation is crucial for assessing large language models' real-world performance beyond flawed academic benchmarks.
• Engineering Reasoning and Instruction (ERI) Benchmark: A Large Taxonomy-driven Dataset for Foundation Models and Agents — The ERI benchmark offers a structured, large-scale dataset for evaluating LLMs in diverse engineering reasoning tasks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Microsoft Research.