Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMs

2026-07-25 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

The SCoRe (Student-Centered one-step Reinforcement) framework addresses the high cost and compounding errors of large language model (LLM) agents by enabling smaller models to achieve expert-level performance. Unlike traditional distillation that imitates full teacher trajectories, SCoRe allows a student LLM to generate its own problem-solving paths, with a larger teacher model intervening only at the first critical error. This approach creates training data specifically matched to the student's evolving abilities and pinpoints weaknesses. The process involves initial supervised fine-tuning (SFT) on corrected trajectories, followed by a reinforcement learning (RL) phase using short-horizon rollouts and targeted key-step rewards. Experiments on 12 challenging benchmarks demonstrate that a 7B-parameter student model, such as Qwen2.5-7B, distilled with SCoRe, matches the agentic performance of a 72B-parameter teacher, Qwen2.5-72B-Instruct, significantly outperforming Behavior Cloning and GRPO.

Key takeaway

For AI Engineers and ML Scientists aiming to deploy cost-effective LLM agents, you should consider adopting the SCoRe framework to distill large models into smaller, high-performing agents. This method significantly reduces the compounding errors inherent in traditional behavior cloning by providing student-centric, targeted corrections and stable reinforcement learning. By leveraging SCoRe, you can achieve agentic capabilities comparable to 72B-parameter models with a 7B-parameter backbone, drastically cutting inference costs and latency for multi-step reasoning and tool-use tasks.

Key insights

SCoRe distills LLM agents by letting students explore, with teachers correcting only the first critical error, reducing error propagation.

Principles

• Student-led exploration improves data alignment.
• One-step teacher correction localizes deficiencies.
• Short-horizon RL stabilizes agent training.

Method

SCoRe initializes with BC, then uses Mentored Problem-Solving where the student explores and the teacher corrects first errors. Finally, RL refinement employs short-horizon rollouts from correct prefixes with key-step rewards.

In practice

• Implement ReAct with CodeAct for deterministic actions.
• Use LLM-as-a-judge for automated answer verification.
• Start RL rollouts from verified correct prefixes.

Topics

• LLM Agents
• Knowledge Distillation
• Reinforcement Learning
• Behavior Cloning
• ReAct Framework
• Code Interpreter

Code references

• modelscope/easydistill

Best for: Research Scientist, NLP Engineer, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• LLM-as-a-Verifier: A General-Purpose Verification Framework — LLM-as-a-Verifier uses scoring token logits to provide fine-grained, scalable verification for agentic tasks without retraining.
• There are at least ten distinct technical families of teacher→student transfer, not one monolithic “distillation.” — Teacher-student transfer involves diverse methods to imbue smaller models with frontier capabilities, driven by economics but fraught with risks.
• MENTOR: Reinforcement Learning via Flexible Teacher-Optimized Rewards for Tool-Use Distillation — Flexible, process-aware reward structures in RL effectively distill LLM tool-use into SLMs, improving out-of-domain generalization over rigid methods.
• A Framework for Evaluating Agentic Skills at Scale — Agent skills significantly enhance LLM behavior and performance, especially for instruction following and making cheaper models competitive.
• Compete Then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond Imitation — AI teacher collaboration excels by building verifiable RL environments, not through direct imitation.
• Context-CoT: Forcing LLMs to Actually Think (No ICL) — LLMs struggle to genuinely learn and reason from novel, complex knowledge provided solely in the prompt, often defaulting to pre-trained biases.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.