ARC Prize 2025 Paper Award 2nd Place SOAR

2025-12-05 · Source: ARC Prize · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

Julian Pel, Cedric Ka, and P.V. from Inria Bordeaux won first runner-up in the ARC Prize 2025 for their paper, "Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC AGI." Their work introduces SOAR, a novel approach that enables Large Language Models (LLMs) to self-improve as evolutionary operators for program synthesis, specifically targeting the challenging ARC AGI benchmark. SOAR alternates between a search phase, where LLMs generate solutions, and a learning phase, where the model is trained on both successful and failed attempts using a technique called hindsight experience replay. This method allows the LLM to learn from diverse data, including "negative" or failed solutions, which enhances its ability to refine programs and generate more diverse outputs, ultimately leading to higher performance without relying on human-engineered data or task-specific DSLs. The team released their datasets to foster further research into the abstractions discovered by the algorithm.

Key takeaway

For AI Scientists and Research Scientists working on challenging program synthesis tasks like ARC AGI, adopting self-improving LLM architectures like SOAR is critical. Your team should explore integrating iterative search-and-learn loops and hindsight experience replay to train models on both successful and failed program generation attempts. This approach can significantly enhance model performance and generalization without requiring extensive human-engineered data, potentially outperforming methods reliant on human-curated DSLs.

Key insights

SOAR enables LLMs to self-improve in program synthesis by learning from both successes and failures through an iterative search and learning loop.

Principles

• Self-improvement through iterative learning is crucial for LLM capabilities.
• Learning from diverse failures enhances model robustness and diversity.
• Program synthesis offers verifiable solutions for AI systems.

Method

SOAR employs an alternating search and learning phase. The search phase generates solutions, while the learning phase fine-tunes the LLM using synthetic data derived from both successful and failed attempts, leveraging hindsight experience replay to improve its role as an evolutionary operator.

In practice

• Utilize hindsight experience replay to learn from failed attempts.
• Alternate between search and learning phases for continuous model improvement.
• Focus on diverse training data to prevent solution collapse.

Topics

• Self-Improving LLMs
• Evolutionary Program Synthesis
• ARC AGI Benchmark
• Hindsight Experience Replay
• Generative AI

Best for: AI Scientist, Research Scientist, AI Researcher, Machine Learning Engineer, AI Student

Related on AIssential

• Unlocking LLM Code Correction with Iterative Feedback Loops — Iterative feedback significantly enhances LLM code correction, with reasoning models excelling at leveraging execution feedback.
• AWS Generative AI Model Agility Solution: A comprehensive guide to migrating LLMs for generative AI production — A structured framework for LLM migration and optimization ensures continuous performance improvement and operational stability.
• ZeroCoder: Can LLMs Improve Code Generation Without Ground-Truth Supervision? — Co-evolving code and test generation with self-generated execution feedback significantly improves LLM performance without ground-truth supervision.
• Francois Chollet + Mike Knoop | ARC Prize @ MIT — ARC benchmarks measure "micro-AGI" properties like efficient interactive learning and goal discovery in novel, small-scale environments.
• SOAR: Self-Correction for Optimal Alignment and Refinement in Diffusion Models — SOAR corrects diffusion model exposure bias by providing dense, on-policy, reward-free supervision for off-trajectory states.
• Papers You Should Know About — AI research is advancing LLM efficiency and agent autonomy through novel architectures, self-improvement mechanisms, and real-world applications.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ARC Prize.