🗞️ Mira Murati’s Thinking Machines made finance expert judgment trainable, beats frontier models with 29.8% fewer errors.

2025-08-21 · Source: Rohan's Bytes · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, FinTech & Digital Financial Services · Depth: Intermediate, medium

Summary

Mira Murati's Thinking Machines developed a model that successfully replicated Bridgewater's private expert judgment in financial tasks, outperforming frontier models with 29.8% fewer errors and achieving 13.8x lower inference cost. The system, which filters finance articles and reports for investor relevance, saw accuracy jump from 46-50% with naive prompts to 74-78% with expert prompts. The breakthrough involved replacing explicit rules with high-quality labels from expert investors, refined through a process where model-disputed cases were reviewed by experts. Training utilized interleaved batches, CISPO loss, and on-policy distillation from stronger teacher checkpoints, enabling the model to learn subtle patterns experts couldn't verbalize. This demonstrates the significant advantage of integrating specialized private judgment over general AI in complex enterprise applications.

Key takeaway

For AI Engineers building specialized enterprise solutions, you should prioritize integrating proprietary expert judgment rather than solely relying on general frontier models. This approach, demonstrated by Thinking Machines, significantly improves accuracy and cost-efficiency in complex domains like finance. Focus on developing robust processes for expert labeling and iterative feedback to train models on nuanced, non-verbalizable patterns, ensuring your AI truly understands domain-specific "taste" and decision-making.

Key insights

Integrating private expert judgment into AI models significantly enhances performance in specialized enterprise tasks, surpassing general frontier models.

Principles

• Bringing private judgment in the loop beats general intelligence.
• High-quality expert labels are crucial for nuanced tasks.
• Iterative expert review refines model learning.

Method

The method involved replacing written rules with expert labels, cleaning labels via expert review, and training with interleaved batches, CISPO loss, and on-policy distillation.

In practice

• Use expert-labeled data for domain-specific AI.
• Implement iterative expert feedback loops.
• Consider CISPO loss for stable RL training.

Topics

• Expert Judgment AI
• Financial AI
• Reinforcement Learning
• CISPO Loss
• On-policy Distillation
• Enterprise AI

Code references

• teamchong/pxpipe

Best for: NLP Engineer, Investor, CTO, AI Engineer, Machine Learning Engineer, Director of AI/ML

Related on AIssential

• Can AI Refute Economic Theory? Evidence from Beyond the Knowledge Cutoff — AI models cannot independently refute economic theory, but human-AI collaboration shows promise for peer review.
• How to build Multi Agents for FINANCE: Outperforming Anthropic — Specialized, locally runnable LLMs can outperform large proprietary models in domain-specific tasks through targeted training and novel evaluation.
• Giving AI the capacity for judgment — Human judgment integrates perception, prediction, and value assessment dynamically, allowing for adaptive, motive-driven understanding beyond AI's fixed scope.
• I need some brutal honesty about the future — AI will transform jobs by automating tasks, requiring humans to master AI tools and apply critical judgment.
• The Brain Trick Behind the World’s Best AI Models — Mixture of Experts decouples model capacity from inference cost by activating only a subset of specialized networks per token.
• Satya Nadella warns that AI could hollow out entire industries, echoing the damage done by globalization — AI risks centralizing value; enterprises must build proprietary learning systems to retain competitive advantage and manage escalating token-based costs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Rohan's Bytes.