AI models are terrible at betting on soccer—especially xAI Grok

2026-04-11 · Source: AI - Ars Technica · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Intermediate, short

Summary

A new study by AI start-up General Reasoning, titled "KellyBench," found that leading AI models from Google, OpenAI, Anthropic, and xAI consistently lost money when betting on the 2023–24 Premier League soccer season. Eight top AI systems were tested in a virtual re-creation, provided with historical data, and instructed to maximize returns and manage risk over a long period. Anthropic’s Claude Opus 4.6 performed best with an average loss of 11 percent, while xAI’s Grok 4.20 went bankrupt in all three attempts. Google’s Gemini 3.1 Pro achieved a 34 percent profit on one try but also experienced bankruptcy. The report highlights AI's struggle with real-world complexity and dynamic, long-horizon tasks, contrasting with its rapid advancements in static tasks like software engineering.

Key takeaway

For AI Product Managers evaluating model deployment in dynamic, real-world environments, this study indicates that current frontier models may underperform significantly compared to their benchmark scores. You should prioritize testing AI systems with long-horizon, adaptive scenarios rather than relying solely on static evaluations to accurately gauge their practical utility and risk.

Key insights

Advanced AI models struggle with dynamic, real-world prediction tasks over extended periods, despite proficiency in static benchmarks.

Principles

• AI performance degrades in dynamic, long-horizon scenarios.
• Static benchmarks misrepresent real-world AI capabilities.

Method

AI agents were given historical soccer data, instructed to build predictive models, and placed bets on match outcomes over a simulated Premier League season to test adaptation.

In practice

• Evaluate AI systems using dynamic, long-horizon benchmarks.
• Recognize AI limitations in complex, real-world decision-making.

Topics

• Large Language Models
• Sports Betting
• AI Benchmarking
• Real-World AI Performance
• KellyBench Report

Best for: Research Scientist, AI Product Manager, AI Scientist, Director of AI/ML, Consultant

Related on AIssential

• What Do Models Still Suck At? - Peter Gostev, Arena.ai, BullshitBench — LLMs still struggle with identifying nonsense and consistently meeting user expectations in complex, real-world tasks.
• I Built an AI Agent that Predicts Match Winners in the ICC Men’s T20 World Cup 2026 — A multi-agent AI system predicts T20 World Cup outcomes by integrating real-time data and contextual reasoning.
• State of AI: May 2026 — Frontier AI now performs offensive cyber operations, while agent performance varies significantly by market type.
• GPT 5.2: OpenAI Strikes Back — AI benchmark performance is increasingly driven by "thinking time" or test-time compute, complicating direct model comparisons.
• GDP.pdf: Can $100B AI Models Master the Documents that Run the World? — Frontier AI models critically fail at processing complex, real-world PDF documents essential for economic and professional workflows.
• not much happened today — AI models are advancing in design, coding, and agentic capabilities, with a focus on efficiency, local deployment, and robust evaluation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI - Ars Technica.