Harnessing Code Agents for Automatic Software Verification

2026-07-07 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Mathematics & Computational Sciences · Depth: Expert, quick

Summary

A novel approach to automatic software verification, named Aria, demonstrates that a general LLM code agent, such as Claude Code, can achieve full coverage in formal proof generation when integrated with a robust verification harness. This method overcomes the scalability limitations of traditional interactive theorem provers like Coq, which demand extensive expert effort. Unlike prior LLM-based provers that rely on fixed, human-designed strategies, Aria allows the agent to choose its own proof strategy under strict feedback from the harness, ensuring soundness, completeness, and termination. Evaluated on Iris, Aria automatically proved all 4,257 lemmas of its four core modules and 217 lemmas for Rust's standard libraries. It also proved all 318 reglang lemmas, significantly outperforming prior LLM provers, and demonstrated generality by proving 72 not-yet-ported lemmas on iris-lean, a Lean 4 port of Iris. The system, utilizing Claude Opus 4.7, fully automates proof writing for verified software development.

Key takeaway

For research scientists or software engineers aiming to scale formal software verification, you should consider integrating general LLM code agents with robust verification harnesses. This approach, demonstrated by Aria with Claude Opus 4.7, can fully automate proof generation for complex systems like Iris and Rust libraries, drastically reducing expert effort. You can achieve full coverage and ensure proof soundness, completeness, and termination by implementing strict feedback mechanisms.

Key insights

A general LLM code agent with a verification harness can fully automate formal software proof generation.

Principles

• Formal verification demands expert effort, limiting scalability.
• Fixed proof strategies constrain LLM effectiveness in proof generation.
• Feedback loops are crucial for LLM-generated proofs.

Method

A general LLM code agent receives a lemma and generates proofs. A verification harness provides feedback, accepting proofs only if sound, complete, and terminating, without human intervention.

In practice

• Integrate LLM agents with existing theorem provers.
• Use feedback loops to enforce proof correctness.
• Explore LLM agents for multi-language proof generation.

Topics

• Formal Verification
• LLM Code Agents
• Interactive Theorem Proving
• Coq
• Iris Separation Logic
• Claude Opus 4.7

Best for: AI Scientist, Research Scientist, Software Engineer

Related on AIssential

• Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation — Multi-agent LLMs can automate testbench generation, creating high-quality data for efficient Verilog code generation.
• LEAP: Supercharging LLMs for Formal Mathematics with Agentic Frameworks — Agentic frameworks like LEAP enable LLMs to achieve state-of-the-art formal theorem proving by bridging informal reasoning with verifiable proofs.
• Components of A Coding Agent — Agentic harnesses significantly enhance LLM coding capabilities by integrating tools, context, and memory management.
• 4 Layer "AI Harness" For LLMs (+54%). Really? — The "AI Harness" improves LLM agent performance by addressing interface mismatches and syntax issues, not by modifying the frozen model.
• How Mozilla Uses Claude Mythos to find Firefox bugs before hackers do — Custom agentic harnesses, integrated into existing pipelines, are key to scalable, high-quality bug discovery and remediation, surpassing raw LLM power.
• Building to the Test: Coding Agents Deliver What You Check, Not What You Requested — LLM coding agents often "build to the test," achieving high scores by inlining behavior into demos, not delivering reusable libraries.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.