HorizonMath: Measuring AI Progress Toward Mathematical Discovery with Automatic Verification

2026-03-16 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, AI for Mathematical Discovery · Depth: Advanced, quick

Summary

HorizonMath is a new benchmark comprising over 100 predominantly unsolved problems across eight domains in computational and applied mathematics, designed to measure AI progress toward mathematical discovery. It includes an open-source evaluation framework for automated verification, focusing on problems where discovery demands significant mathematical insight but verification is computationally efficient. This design makes HorizonMath robust against data contamination, as solutions are unknown, and current state-of-the-art models typically score near 0%. Unlike existing research-level benchmarks that rely on expensive formal proof verification or manual review, HorizonMath offers scalable automated verification. Initial evaluations using this platform show that GPT 5.4 Pro proposed solutions for two problems that potentially improve upon the best-known published results, suggesting novel contributions awaiting expert review. HorizonMath is released as an open challenge and a community resource.

Key takeaway

For AI researchers focused on advancing mathematical reasoning, HorizonMath offers a unique, contamination-immune benchmark to test models on unsolved problems. Your team should consider integrating HorizonMath into your evaluation pipeline to identify true discovery capabilities, as GPT 5.4 Pro has already shown potential for novel contributions on this platform.

Key insights

HorizonMath benchmarks AI's ability to solve unsolved math problems with automated, scalable verification.

Principles

• Discovery is hard, verification can be simple.
• Unknown solutions prevent data contamination.

Method

HorizonMath evaluates AI on unsolved math problems using an open-source framework for automated, computationally efficient verification, bypassing expensive formal proof or manual review.

In practice

• Use HorizonMath for novel AI math research.
• Contribute solutions to expand the benchmark.

Topics

• Mathematical Discovery
• AI Benchmarking
• Large Language Models
• Automated Verification
• Computational Mathematics

Best for: AI Researcher, AI Scientist, Research Scientist

Related on AIssential

• First Proof Second Batch — Current AI systems are being rigorously tested on research-level mathematics problems to assess their problem-solving capabilities.
• ERBench: A Benchmark and Testsuite for Equation Discovery Algorithms — Equation recovery is a superior metric for evaluating true model discovery and generalization in symbolic regression.
• Formal Conjectures: An Open and Evolving Benchmark for Verified Discovery in Mathematics — Formal Conjectures is an open, evolving benchmark for evaluating automated reasoning on research-level mathematics using Lean 4.
• Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery — AI for mathematical reasoning is a rapidly evolving frontier, integrating diverse techniques for verified discovery.
• Riemann-Bench: A Benchmark for Moonshot Mathematics — Research-level mathematics remains largely beyond current AI capabilities, despite strong performance on competition problems.
• How did Gpt solve the erdos problem? A demonstration — AI's problem-solving success stems from uncovering hidden algebraic structures, not just surface patterns.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.