INFUSER: Influence-Guided Self-Evolution Improves Reasoning

2026-06-08 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, quick

Summary

INFUSER is an iterative co-training framework that significantly improves the reasoning capabilities of pretrained language models through self-evolution, requiring minimal external supervision. It comprises a Generator, which drafts questions and golden answers from unstructured documents, and a Solver, which trains on this data. The Solver is rewarded for correctness, while the Generator is rewarded by an optimizer-aware influence score, measuring a question's utility in improving the Solver on the target distribution. INFUSER introduces DuGRPO, a dual-normalized GRPO variant, for Generator training to manage noisy influence scores, creating an adaptive curriculum. On Qwen3-8B-Base, INFUSER achieves over 20% relative improvement against strong self-evolution baselines on Olympiad and SuperGPQA benchmarks. An 8B INFUSER co-evolving generator also outperforms a frozen 32B thinking generator on math and coding tasks, demonstrating the framework's effectiveness and generalizability.

Key takeaway

For Machine Learning Engineers developing self-evolving language models, INFUSER provides a powerful framework to significantly boost reasoning capabilities. You should consider adopting its influence-guided co-training approach, particularly the DuGRPO variant, to create adaptive curricula that directly improve your solver models. This method allows you to build more robust reasoning systems, like those outperforming 32B models with an 8B generator, by focusing on questions most beneficial for current model improvement.

Key insights

Influence-guided self-evolution with co-training and adaptive curricula significantly improves language model reasoning.

Principles

• Reward generators by solver improvement.
• Co-training roles enhance self-evolution.
• Adaptive curricula optimize learning paths.

Method

INFUSER co-trains a Generator (drafts questions/answers) and a Solver (trains on them). Generator reward uses optimizer-aware influence scores, processed by DuGRPO, to form an adaptive curriculum.

In practice

• Implement DuGRPO for noisy reward signals.
• Employ co-evolving generators for math/coding.
• Extend framework to finetuned LLMs.

Topics

• Self-evolution
• Language Model Reasoning
• Co-training Frameworks
• DuGRPO
• Influence Scores
• Adaptive Curriculum

Code references

• FFishy-git/INFUSER

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning — EvoTrainer co-evolves LLM policies and training harnesses, moving beyond static recipe search for agentic RL.
• Teacher-Free Self-Training Amplifies but Does Not Compound: A Pass@$K$ Crossover on a Free-Verifier Domain — Self-training amplifies existing language model capabilities by concentrating probability mass, not by expanding reach.
• Claude Code + Karpathy's NEW Self-Evolving System = 10x Code Generation — AI agents can build and maintain self-evolving knowledge systems using structured wikis and schema rules.
• Self-Improving Language Models with Bidirectional Evolutionary Search — Bidirectional Evolutionary Search (BES) enhances language model self-improvement by coupling forward candidate evolution with backward goal decomposition.
• LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance — LANG enhances multilingual LLM reasoning by using language-adaptive hints with a progressive decay schedule, preventing language drift.
• GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models — GIFT enhances instruction-tuned models by guiding base model adapter training with instruction model confidence signals.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.