Curriculum Learning for Efficient Chain-of-Thought Distillation via Structure-Aware Masking and GRPO

2026-07-11 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, extended

Summary

BRIDGE is a three-stage curriculum learning framework designed to distill Chain-of-Thought (CoT) reasoning from large language models into smaller student models like Qwen2.5-3B-Base. It addresses the capacity mismatch where verbose teacher rationales overwhelm compact models. The framework first builds structural understanding via masked shuffled reconstruction, then optimizes for accuracy and brevity using Group Relative Policy Optimization (GRPO) on masked completion tasks. Finally, it guides the student to internalize teacher knowledge for difficult cases through targeted rewriting, also optimized with GRPO. Experiments on GSM8K show Qwen2.5-3B-Base achieved an 11.29% accuracy improvement, reaching 76.19%, while reducing output length by 27.4% to 167 tokens, outperforming prior distillation methods.

Key takeaway

For Machine Learning Engineers deploying smaller language models in resource-constrained environments, BRIDGE offers a robust method to achieve high-quality Chain-of-Thought reasoning with significantly reduced output verbosity. You should consider implementing this three-stage curriculum, particularly its hierarchical reward structure and teacher-guided internalization, to overcome capacity mismatch challenges and improve both accuracy and inference efficiency for models like Qwen2.5-3B-Base. This approach helps avoid common pitfalls like repetition loops and logical degradation.

Key insights

A curriculum learning framework enables small models to distill and compress Chain-of-Thought reasoning effectively.

Principles

• Effective CoT distillation requires structural understanding.
• Prioritize correctness before efficiency in rewards.
• Students can compress reasoning better than generate it.

Method

BRIDGE uses a three-stage curriculum: 1) structure-aware warmup via masked shuffled reconstruction, 2) GRPO-based compression on masked completion, and 3) teacher-guided internalization for hard cases.

In practice

• Apply masked shuffled reconstruction for structural pre-training.
• Implement hierarchical rewards for accuracy and brevity.
• Target difficult samples with teacher-scaffolded rewriting.

Topics

• Chain-of-Thought Distillation
• Curriculum Learning
• Reinforcement Learning
• LLM Compression
• Mathematical Reasoning
• Qwen2.5-3B-Base

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Knowledge Distillation from Large Reasoning Models to Compact Student Models: A Case Study on the John O Bryan Mathematics Competition — Chain-of-Thought distillation significantly improves compact models' mathematical reasoning, with response length being a critical factor.
• Hierarchical Chain-of-Thought: Enhancing LLM Reasoning Performance and Efficiency — Hierarchical Chain-of-Thought (Hi-CoT) improves LLM reasoning accuracy and efficiency by enforcing structured, alternating instruction-execution steps.
• DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Models — Block-size curriculum learning enables efficient diffusion models to achieve strong long chain-of-thought reasoning by bridging granularity gaps.
• Context-CoT: Forcing LLMs to Actually Think (No ICL) — LLMs struggle to genuinely learn and reason from novel, complex knowledge provided solely in the prompt, often defaulting to pre-trained biases.
• Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning — Improving initial rollout distribution is crucial for efficient self-reasoning distillation in large language models.
• Multilingual Chain-of-Thought Compression via Cross-Lingual Distillation — MCD compresses multilingual Chain-of-Thought reasoning using cross-lingual data alignment and reinforcement learning, improving efficiency and low-resource language robustness.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.