Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning

2026-07-20 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, extended

Summary

Bootstrapped Iterative Self-Reasoning Distillation (BIRD) is a two-stage method designed to compress Chain-of-Thought (CoT) traces in large language models, addressing the inefficiency of long, redundant reasoning. It tackles the "cold-start prefix-support bottleneck" in existing on-policy self-distillation (OPSD) by improving the initial rollout distribution. BIRD first employs a lightweight prompt-switch Supervised Fine-Tuning (SFT) step, using correctness-filtered concise traces generated by the base model under a brevity instruction. This converts instruction-induced conciseness into a default reasoning behavior. Subsequently, BIRD applies on-policy reverse-KL distillation from this "warm" model, ensuring supervision is applied to cleaner, more informative prefixes. Experiments on Qwen3-8B show BIRD improves MATH-500 accuracy from 86.2% to 92.0% while reducing average response length from 3,099 to 1,115 tokens. It also generalizes to DeepSeek-R1-Distill-Llama-8B, enhancing accuracy by 8.0 percentage points and shortening responses by 33%.

Key takeaway

For Machine Learning Engineers optimizing large language model inference for complex reasoning tasks, you should consider implementing a two-stage distillation approach like BIRD. By first warm-starting your model with lightweight SFT on correctness-filtered, concise traces, you can establish a more efficient reasoning baseline. This initial step significantly improves the prefix support for subsequent on-policy self-distillation, leading to substantially shorter response lengths and higher accuracy than traditional cold-start methods.

Key insights

Effective on-policy self-distillation for reasoning compression requires optimizing the prefix support where supervision is applied.

Principles

• Teacher effectiveness relies on queried rollout prefixes.
• Instruction-induced conciseness can become default behavior.

Method

BIRD uses a two-stage process: first, offline SFT with correctness-filtered, prompt-switched concise traces, then on-policy reverse-KL self-distillation.

In practice

• Pre-train with lightweight SFT bootstrap.
• Filter self-generated traces for correctness.
• Use prompt-switching for conciseness internalization.

Topics

• Large Language Models
• Chain-of-Thought Reasoning
• Self-Distillation
• Reasoning Compression
• Supervised Fine-Tuning
• LoRA Adaptation

Code references

• KawhiC/BIRD

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning — Improving initial rollout distribution is crucial for efficient self-reasoning distillation in large language models.
• Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces — Large reasoning models employ a two-phase, coarse-to-fine mechanism for multi-label tasks, which can be mechanistically distilled.
• LoRi: Low-Rank Distillation for Implicit Reasoning — Hidden-state reasoning trajectories exhibit low-rank structure, enabling efficient distillation for implicit reasoning.
• H$^2$SD: Hybrid Hindsight Self-Distillation — H^2SD improves LLM reasoning by adaptively using a teacher for either update modulation or explicit correction based on trajectory success.
• Knowledge Distillation from Large Reasoning Models to Compact Student Models: A Case Study on the John O Bryan Mathematics Competition — Chain-of-Thought distillation significantly improves compact models' mathematical reasoning, with response length being a critical factor.
• Memory Sparse Attention to Get to 100 Million Tokens in Context — Self-distillation can harm reasoning by suppressing uncertainty, while sparse attention enables extreme context scaling.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.