ARC Prize 2025 Paper Award 3rd Place ARC without Pretraining

2025-12-05 · Source: ARC Prize · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

Isaac Liao, a second-year student at Carnegie Mellon University, won third place and a $5,000 prize in the ARC Prize 2025 paper award for his work "ARC AGI without Pre-training." His research, rooted in information compression and the Minimum Description Length (MDL) principle, proposes "Compress Arc," a novel approach that trains a neural network to overfit individual ARC AGI puzzles. This method compresses puzzle data into neural network weights, enabling generalization by hardcoding these weights into a program to reproduce solutions. Compress Arc achieves approximately 34% accuracy on the ARC AGI1 training set and 20% on its evaluation set, with 4.17% on ARC AGI2's evaluation set. A key innovation is making the discrete description length differentiable for gradient descent through "seed manipulation," which injects information into random variables to control their emergence. Liao's work demonstrates that many successful machine learning architectures and loss functions are implicitly compressive.

Key takeaway

For AI scientists and machine learning engineers working on data-constrained problems like ARC AGI, consider exploring information compression and the Minimum Description Length (MDL) principle. Your approach could yield strong generalization without massive pre-training, as demonstrated by Compress Arc. Focus on designing architectures with specific inductive biases and leverage techniques like seed manipulation to make discrete optimization problems amenable to gradient descent, potentially uncovering new algorithmic derivations.

Key insights

Information compression, particularly the MDL principle, offers a powerful lens for developing generalizable AI solutions without extensive pre-training.

Principles

• Compression implies intelligence.
• Minimize description length for generalization.
• MDL can guide architecture design.

Method

Compress Arc uses a neural network to overfit individual ARC AGI puzzles, compressing puzzle data into network weights. These weights are then hardcoded into a program to reproduce solutions, leveraging differentiable description length via seed manipulation.

In practice

• Explore MDL for data-constrained problems.
• Consider seed manipulation for differentiable discrete values.
• Engineer inductive biases into network layers.

Topics

• ARC Prize 2025
• Minimum Description Length
• Information Compression
• ARC AGI Benchmark
• Compress Arc

Best for: AI Scientist, Machine Learning Engineer, AI Student

Related on AIssential

• Francois Chollet + Mike Knoop | ARC Prize @ MIT — ARC benchmarks measure "micro-AGI" properties like efficient interactive learning and goal discovery in novel, small-scale environments.
• 84.0% on ARC-AGI2 (840/1000) using LLM program synthesis + deterministic verification — no fine-tuning, no neural search — Combining LLM program synthesis with deterministic verification significantly boosts performance on complex reasoning tasks.
• [AINews] Humanity's Last Gasp — AI's progress in automation paradoxically correlates with increased human workload, highlighting a gap in generalizable learning.
• ARC-AGI-3 — General intelligence is defined by the ability to learn new skills, not just perform existing ones.
• CompreSSM: Compressing State-Space Models During Training with Hankel Singular Values — CompreSSM uses Hankel singular values to prune state-space models during training, achieving faster, leaner models without accuracy loss.
• Why AI Needs Better Benchmarks — AI benchmarks must continuously evolve to measure true general intelligence beyond memorization and narrow task performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ARC Prize.