How a 20-Person Startup Won Gold at the Math Olympiad—Tying With OpenAI & DeepMind (Tudor Achim, CEO of Harmonic)
Summary
Harmonic, a 20-person startup, achieved a gold medal at the International Math Olympiad, solving five of six problems and tying with OpenAI and Google DeepMind. Their mathematical agent, Aerosol, is the world's first AI to produce formally verified outputs, ensuring 100% correctness through the Lean 4 programming language. This approach contrasts with traditional LLMs by integrating hallucination for creative problem-solving with rigorous, computer-checkable proofs. CEO Tudor Achim highlights Lean's Mathlib as a key enabler, fostering a network effect for open-source math. Harmonic's technology aims to revolutionize high-stakes software development by making verified code standard, moving beyond the "trust me bro" principle of unverified AI outputs. Achim predicts AI mathematicians will surpass human capabilities in specific tasks within two to three years.
Key takeaway
For AI Engineers and Directors of AI/ML developing high-stakes systems, you should prioritize integrating formal verification into your AI development pipeline. Relying solely on probabilistic LLM outputs for critical applications like finance, cybersecurity, or medical devices introduces unacceptable risk. Adopting tools like Harmonic's Aerosol, which provides computer-checkable proofs, ensures 100% correctness, drastically reducing bugs and undefined behavior, and establishing a new standard for trustworthy AI.
Key insights
Formal verification, powered by Lean, is crucial for building trustworthy and scalable AI systems capable of advanced mathematical reasoning.
Principles
- Hallucination is key to creativity and reasoning
- Math is fundamentally search and pattern recognition
- Formal verification makes AI outputs useful by construction
Method
Aerosol combines LLM-driven hallucination for exploring mathematical ideas with formal verification using Lean to check proofs step-by-step, leveraging reinforcement learning on synthetic data for continuous self-improvement.
In practice
- Validate mathematical solutions from other AIs
- Develop formally verified software theories
- Model complex quantum physics problems
Topics
- Formal Verification
- Mathematical AI
- Lean Programming Language
- Aerosol Agent
- Reinforcement Learning
- Software Verification
- AI Trustworthiness
Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, AI Engineer, Director of AI/ML
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by The Generalist.