How Post-Training Shapes Biological Reasoning Models

2026-06-15 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Life Sciences & Biology, Research Methodology & Innovation · Depth: Expert, quick

Summary

Scientific reasoning models for biology, which combine language models with foundation models trained on multimodal biological data like DNA, RNA, and proteins, are built through post-training. A study trained and evaluated over 100 such models across genomics, transcriptomics, and proteins, varying backbone, continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL). Measuring both in-domain (ID) and out-of-domain (OOD) performance, the research found that each post-training stage distinctly reshapes generalization. CPT improves downstream performance by aligning models with biological language. SFT consistently increases ID performance but causes OOD performance to peak early and decline. RL, when applied to strong SFT checkpoints with aligned rewards, improves OOD performance and partially recovers generalization. The findings indicate that biological reasoning performance depends on how training stages are composed, not merely on additional supervision or compute. Optimal ID-OOD trade-offs under fixed budgets involve brief SFT, larger RL allocations, and asymmetric adaptation capacity across stages.

Key takeaway

For AI Scientists and Machine Learning Engineers optimizing biological reasoning models, understand that generalization is not a monotonic gain. You should compose post-training stages strategically, prioritizing brief supervised fine-tuning (SFT) for initial in-domain performance. Allocate larger resources to reinforcement learning (RL) after SFT to recover and improve out-of-domain generalization. Design asymmetric adaptation capacities across these stages to achieve the strongest in-domain/out-of-domain trade-off, rather than simply adding more compute.

Key insights

Post-training stages distinctly shape biological reasoning model generalization, requiring careful composition for optimal performance.

Principles

• Post-training stages reshape generalization distinctly.
• SFT boosts in-domain, but can degrade out-of-domain.
• RL can recover out-of-domain generalization post-SFT.

Method

The study trained and evaluated over 100 biological reasoning models, varying backbone, CPT, SFT, and RL, measuring in-domain and out-of-domain performance.

In practice

• Prioritize brief SFT for ID-OOD trade-off.
• Allocate larger budgets to RL post-SFT.
• Design asymmetric adaptation capacity across stages.

Topics

• Biological Reasoning Models
• Post-Training Optimization
• Continued Pre-training
• Supervised Fine-Tuning
• Reinforcement Learning
• Model Generalization

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability — Reasoning SFT can generalize cross-domain, but it is conditional on optimization, data quality, and model capability.
• PriFT: Prior-Support Guided Supervised Fine-Tuning — PriFT improves SFT generalization by reweighting tokens based on a frozen pretrained model's "prior support" for stability.
• Learning What to Learn: Stage-Specific Data Sets for SFT-then-RL in Small Language Model Reasoning — Data strategy for SFT-then-RL pipelines should align with each stage's distinct role in skill acquisition and consolidation.
• On Asymmetric Optimization of Reasoning and Perception in Vision-Language Model Post-Training — VLM post-training creates a perception-reasoning asymmetry due to token imbalance (SFT) or reward coupling (RL), hindering end-to-end performance.
• In-Context Learning for Latent Space Bayesian Optimization — Adapting tabular foundation models for latent space Bayesian optimization requires specific pretraining to address domain mismatches.
• Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning — Larger LLMs demonstrate superior efficiency and performance in RL post-training for mathematical reasoning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.