Train separately, merge together: Modular post-training with mixture-of-experts

2026-04-20 · Source: Ai2 Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, medium

Summary

Released on April 20, 2026, by Ai2 researchers Jacob Morrison, Sanjay Adhikesaven, Akshita Bhagia, Matei Zaharia, Noah A. Smith, and Sewon Min, BAR (Branch-Adapt-Route) is a novel recipe for modular post-training of language models. It addresses the challenges of updating or extending models after initial post-training stages, which typically involve expensive full retraining or risk losing existing capabilities. BAR sidesteps these issues by training independent domain experts—such as for math, code, tool use, and safety—each through its own complete pipeline, then composing them into a unified Mixture-of-Experts (MoE) architecture. A key technical contribution is a progressive unfreezing schedule for shared parameters across mid-training, supervised fine-tuning (SFT), and reinforcement learning with verified rewards (RLVR) stages. BAR achieves strong performance, outperforming baselines that don't require rerunning mid-training from scratch (49.1 vs. 47.8 overall) and enabling independent expert upgrades with linear cost scaling, such as a +16.5 point improvement in code performance by replacing an expert.

Key takeaway

For Machine Learning Engineers iterating on large language model capabilities, BAR provides a robust solution to update specific domains without costly full retraining. If you are struggling with catastrophic forgetting or the quadratic cost of monolithic updates, consider adopting this modular MoE approach. It allows you to upgrade individual experts, like improving code performance by +16.5 points, with linear cost scaling and minimal impact on other domains.

Key insights

Modular post-training with progressive unfreezing and MoE architecture enables flexible, cost-effective language model updates.

Principles

• Post-training demands progressive shared parameter unfreezing.
• Domain experts require mixed general and specific SFT data.
• Averaging diverged shared parameters maintains performance.

Method

BAR trains independent domain experts with progressive unfreezing, merges them by averaging shared parameters, then trains a lightweight router on a 5% SFT data sample.

In practice

• Implement progressive unfreezing for attention in RL.
• Combine domain-specific with general SFT data.
• Average shared parameters when merging MoE experts.

Topics

• Mixture-of-Experts
• Modular Training
• Post-training
• Progressive Unfreezing
• Language Models
• Model Upgrades

Code references

• allenai/FlexOlmo

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, Machine Learning Engineer, Director of AI/ML

Related on AIssential

• DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts — DAG-MoE improves Mixture-of-Experts performance by replacing weighted-summation with learned structural aggregation, expanding expert-combination space and enabling multi-step reasoning.
• SPRI: SVD-Partitioned Residual Initialization for Data-Constrained MoE Upcycling — SPRI leverages SVD-partitioned residuals from pretrained FFNs to create diverse MoE experts, improving upcycling under data constraints.
• Generic Expert Coverage for Pruning SparseMixture-of-Experts Language Models — Preserving cross-corpus expert coverage with generic data improves MoE pruning without downstream calibration.
• Mixture of Experts: The Architecture Behind Today’s Largest AI Models — Mixture of Experts decouples model knowledge capacity from per-input computational cost, enabling massive scale.
• Post-Training 101: From Base Model to Assistant — Post-training transforms base LLMs into assistants by teaching them conversational behavior and response preferences.
• Leveraging Routing Dynamics in Mixture-of-Experts Models for Efficient Language Adaptation — MoE language specialization emerges in final layers, enabling efficient adaptation by targeting those layers.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Ai2 Blog.