ARC Prize 2025 Top Score 2nd Place the ARChitects

2025-12-05 · Source: ARC Prize · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Data Science & Analytics · Depth: Advanced, extended

Summary

The "Architects" team, comprising Yantisov, Daniel Fransson, and David Hartman, secured second place and a $10,000 prize in the ARC Prize 2025 competition, also earning an honorable mention for their paper "Product of Experts with LLM's: Boosting Performance on ARC is a Matter of Perspective." This marks their second consecutive win, having also won ARC Prize 2024. Their 2025 approach involved a two-pronged strategy: optimizing their previous year's solution with speculative decoding and prefix caching for a five-fold speed increase, and exploring novel methods like masked diffusion LLMs. They utilized a Llama-based masked diffusion model (Lada) with a 2D "golden gate" positional encoding for better alignment with ARC's grid tasks. The team also experimented with synthetic data generation using fine-tuned Qwen 2.5 Coder and a vision LLM as a reward, though this proved inefficient. A key innovation was using the LLM not only as a generator but also as a scorer, and a recursive sampling method that back-fed model output as input, running 30-40 iterations until convergence, even though the model was not explicitly trained this way. They also rediscovered and applied the "Product of Experts" approach for combining probabilities from different perspectives.

Key takeaway

For AI and Research Scientists developing solutions for constrained environments like ARC, consider a dual approach: optimize existing high-performing models while simultaneously exploring novel architectures and sampling methods. Your team should prioritize early and frequent Kaggle submissions to debug environmental challenges and validate approaches. The "Product of Experts" concept, combining probabilities from multiple model perspectives, can significantly boost solution selection accuracy, especially when dealing with diverse candidate solutions.

Key insights

The Architects team achieved top ARC Prize scores through iterative optimization, novel LLM architectures, and recursive sampling techniques.

Principles

• Iterate on existing solutions for competitive advantage.
• Combine diverse model perspectives for robust scoring.
• Leverage competition timelines for intense productivity.

Method

The team optimized a Llama-based masked diffusion LLM with 2D positional encoding, applied speculative decoding and prefix caching, and developed a recursive sampling method that feeds model output back as input for iterative refinement.

In practice

• Implement speculative decoding for faster inference.
• Utilize prefix caching to accelerate scoring.
• Explore recursive model feedback for solution refinement.

Topics

• ARC Prize
• Large Language Models
• Masked Diffusion LLMs
• Product of Experts
• Recursive Sampling

Best for: AI Scientist, Research Scientist, AI Researcher, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines — Selector quality, not just team diversity, dictates multi-agent LLM pipeline performance.
• ARC Prize 2025 Top Score 3rd Place MindsAI — Integrating psychology with AI led to novel methods for evaluating and enhancing large language models' generalization.
• 2025 Interconnects year in review — DeepSeek's open-weight, efficient MoE and MLA architectures challenge AI's cost curve, intensifying the global AI race and geopolitical tech competition.
• True Positive Weekly #152 — The AI landscape is rapidly evolving with advancements in model architectures, training methodologies, and practical applications.
• ChatGPT Wrote, Claude Reasoned, DeepSeek Critiqued — and I Somehow Published a Book in 3 Weeks — Different AI models excel at distinct tasks, enabling a specialized "AI product team" approach to content creation.
• Modality-Driven Search with Holistic Trace Judging for ARC-AGI-2 — Diverse modality-driven candidate generation combined with holistic trace judging improves LLM performance on abstract reasoning by recovering minority correct answers.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ARC Prize.