CurateEvo: Data-Curation Evolving for Agentic Post-Training

2026-07-08 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

CurateEvo is a novel failure-driven dynamic evolution framework designed to improve data curation for Large Language Model (LLM) agents during post-training. Unlike traditional methods that treat data curation as a static preprocessing step, CurateEvo addresses the challenge of enhancing long-horizon decision making by dynamically adapting curation strategies based on environment feedback and observed failures. The framework represents its curation strategy as executable code, which is iteratively rewritten using failed trajectories from a held-out development set. In each epoch, the evolved strategy transforms a raw corpus into supervised fine-tuning data, reinforcement learning data, and an inference-time memory bank. This process first boosts effectiveness by diagnosing recurring failure modes and adjusting data, then improves efficiency by pruning low-utility training turns with a cost-aware objective. Experiments on ACEBench-Agent, BFCL-V4, and τ²-Bench demonstrated CurateEvo's superior performance, yielding average score improvements of 3.2 and 2.7 points, while also reducing curation overhead.

Key takeaway

For Machine Learning Engineers optimizing LLM agent post-training, you should consider implementing dynamic, failure-driven data curation. CurateEvo's approach of iteratively refining data based on observed failures can significantly improve long-horizon decision making and reduce curation overhead. Evaluate your current fixed data pipelines and explore methods to integrate adaptive data augmentation, filtering, and refinement to boost agent performance and efficiency.

Key insights

CurateEvo dynamically refines LLM agent training data based on observed failures, improving long-horizon decision making.

Principles

• Data curation should be dynamic.
• Failure modes drive data refinement.
• Cost-aware objectives enhance efficiency.

Method

CurateEvo represents curation strategy as executable code, iteratively rewriting it using failed trajectories from a dev set. This strategy transforms raw data into SFT, RL, and memory bank data, optimizing for effectiveness and efficiency.

In practice

• Implement dynamic data curation.
• Diagnose agent failure modes.
• Prune redundant training data.

Topics

• LLM Agents
• Data Curation
• Post-Training Optimization
• Failure-Driven Learning
• Reinforcement Learning
• Supervised Fine-Tuning

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-training — GRACE curates reasoning data by scoring individual steps based on gradient alignment and trajectory consistency.
• MetaEvo: A Meta-Optimization Framework for Experience-Driven Agent Evolution — MetaEvo enables LLM agents to continually improve reasoning by optimizing how they learn principles from experience, not just what they store.
• Explainable Iterative Data Visualisation Refinement via an LLM Agent — An LLM-driven agent automates and explains data visualization refinement by iteratively optimizing DR hyperparameters.
• Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning — Larger LLMs demonstrate superior efficiency and performance in RL post-training for mathematical reasoning.
• Are LLMs safe? — LLM training data curation, especially via quality filters, embeds implicit biases that shape model behavior and capabilities.
• RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agents — RODS dynamically synthesizes data by detecting capability boundaries, resolving sample depletion in multi-turn tool-use RL.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.