Metaprogramming, Synthesis & Verification • Nada Amin • YOW! 2025

2026-06-02 · Source: GOTO Conferences · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, extended

Summary

Nada Amin's presentation explores metaprogramming, synthesis, and verification, emphasizing the critical role of formal verification in ensuring the correctness of LLM-generated code. The talk introduces Daphne, a verification language used to define design-by-contract specifications and formally prove program properties, as demonstrated with arithmetic expression evaluation and optimization. Daphne's semi-automation, based on decades of research, has been applied in safety-critical systems like AWS's authorization engine, which handles a billion invocations per second. The discussion extends to orchestrating LLMs with verifiers using Monte Carlo Tree Search for verified synthesis, addressing challenges like the "scaffolding paradox" and the need for high-quality edits. It also delves into constraint solving with Holi for synthesizing values and expressions, and relational programming for generating code from high-level specifications, advocating for strong guarantees in future code generation.

Key takeaway

For AI Engineers developing code with large language models, integrating formal verification tools like Daphne into your workflow is crucial. This ensures strong correctness guarantees, reduces manual review burdens, and guides development by identifying bugs early with counter-examples. Consider adopting verification-aware languages and exploring neuro-symbolic orchestration methods to enhance the reliability and trustworthiness of your generated code, particularly for safety-critical applications.

Key insights

Integrating formal verification with LLM-driven code synthesis is essential for ensuring strong guarantees and correctness.

Principles

• Verification guides development and reduces manual code review burden.
• Optimistic verifier use ensures guardrails for generated code.
• Relational programming enables synthesis from high-level specifications.

Method

Monte Carlo Tree Search orchestrates LLMs and verifiers by using verifier feedback in the evaluation phase to guide program synthesis, ensuring correctness at each step.

In practice

• Use Daphne for formal verification of design-by-contract specifications.
• Employ Daphne Sketcher to debug code with counter-examples.
• Apply Holi for Python puzzle solving via constraint satisfaction.

Topics

• Formal Verification
• Program Synthesis
• Large Language Models
• Daphne Verification Language
• Neuro-symbolic AI
• Relational Programming

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, AI Engineer, AI Architect

Related on AIssential

• Your AI Just Wrote 500 Lines of Code. Can You Prove Any of It Works? — A framework classifies AI-generated code by its formal verifiability based on five structural properties.
• Our MongoDB TLA+ Workshop — AI tools like Claude Code make formal methods like TLA+ more accessible and feasible for system design and verification.
• How a 20-Person Startup Won Gold at the Math Olympiad—Tying With OpenAI & DeepMind (Tudor Achim, CEO of Harmonic) — Formal verification, powered by Lean, is crucial for building trustworthy and scalable AI systems capable of advanced mathematical reasoning.
• Scaling AI Beyond Informal: Axiom Math's Carina Hong - StartupHub.ai — Formal verification grounds AI in mathematical rigor, enabling reliable, predictable, and collaborative scaling beyond informal, heuristic-based methods.
• How to vibe code in science: early adopters share their tips — Vibe coding with LLMs enables rapid code generation and application development through conversational prompts, democratizing complex programming tasks.
• MIT Prof. Explains AI-Assisted Programming: Part 2 — Combining LLMs with symbolic reasoning enhances code reliability and addresses ethical challenges in AI programming.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by GOTO Conferences.