Stop Making Models Bigger, Make Them Behave — Kobie Crawdord, Snorkel

2026-06-10 · Source: AI Engineer · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, long

Summary

Snorkel's research, in partnership with UC Berkeley's RLLM team, demonstrated that a 4 billion parameter model could outperform a 235 billion parameter model on a financial analysis tool-use task. This was achieved through high-quality data generation and Reinforcement Learning (RL) training, costing under \$500 per 21-hour run. The smaller model, fine-tuned in a self-contained FinQA environment, learned crucial tool-use discipline, such as querying available tables and inspecting schemas, and self-corrected errors. This contrasts with the larger model's failure to use tools and subsequent hallucination. Surprisingly, training with only single-table questions yielded the best performance uplift, even generalizing to multi-table reasoning tasks, doubling performance from 13.9% to 26.6%.

Key takeaway

For Machine Learning Engineers deploying enterprise-grade LLMs, if you are struggling with large model inference costs or data control, consider targeted Reinforcement Learning with high-quality, behavior-specific data. Focus on diagnosing and training for precise tool-use behaviors, as this can enable smaller, more efficient models to achieve superior performance and reliability in production environments.

Key insights

Focused RL training with high-quality, behavior-specific data enables smaller models to surpass larger ones in tool-use tasks.

Principles

• Tool-use discipline is more critical than raw reasoning for specific tasks.
• Smaller models can achieve large model performance with targeted RL.
• The "Terence Tao effect" highlights over-engineering with overly large models.

Method

Generate expert-curated, high-quality data, then apply GRPO-based RL training within a self-contained environment like FinQA, focusing on specific behavioral improvements.

In practice

• Build evaluation rubrics to diagnose specific model failure modes.
• Prioritize data quality and expert-in-the-loop data generation.
• Consider single-table training for broader tool-use generalization.

Topics

• Reinforcement Learning
• LLM Tool Use
• Data Quality
• Financial AI
• Model Efficiency
• FinQA Environment

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, Director of AI/ML

Related on AIssential

• Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments — Automated environment generation and verifiable rewards significantly boost LLM tool-use performance via reinforcement learning.
• RL Scaling Laws for LLMs — Scaling laws, while precise for LLM pretraining, are more complex and context-dependent in reinforcement learning.
• Tool-Aware Optimization with Entropy Guidance for Efficient Agentic Reinforcement Learning — TAO-RL stabilizes agentic reinforcement learning by filtering tool-use trajectories and guiding exploration with entropy for efficient LLM policy optimization.
• Should You Use Your Large Language Model to Explore or Exploit? — LLMs underperform in exploitation tasks but effectively explore large, semantically rich action spaces.
• Capability Self-Assessment: Teaching LLMs to Know Their Limits — Reinforcement learning effectively teaches large language models to recognize their limitations, outperforming supervised fine-tuning.
• Automated Optimization Modeling via a Localizable Error-Driven Perspective — Localized error patterns in optimization modeling enable targeted data synthesis and refined post-training for LLMs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Engineer.