Agentic Proof and Property-Based Testing via Property-Templates in Data-Intensive Computing

2026-07-13 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Data Science & Analytics · Depth: Expert, extended

Summary

The paper introduces "property templates" for validating correctness properties in data-intensive computing systems like Apache Spark. It addresses the bottleneck of intent specification and validation in AI-driven code generation. The proposed "DualVeri" framework uses property templates to guide both formal verification in Lean 4 and property-based testing (PBT) on real implementations. Evaluation shows property templates increase agentic proof engineering success by up to 2.6× (avg: 1.6×) and reduce proof hallucinations by 59%. Template-guided PBT synthesis reduces intent misalignments from 22 to 1 and cuts synthesis cost by up to 5.7× (avg: 3.8×). It also exceeds a state-of-the-art Spark fuzzer in code coverage. The dual-track approach provides strong evidence and diagnostic value, identifying model-implementation gaps.

Key takeaway

For AI Scientists and Machine Learning Engineers developing or validating data-intensive systems, adopting property templates significantly enhances the reliability and efficiency of correctness validation. You should integrate template-guided formal proof (e.g., Lean 4) and property-based testing (e.g., PySpark) into your workflow. This approach reduces LLM-driven hallucination and synthesis costs, while cross-validation helps you pinpoint critical model-to-implementation discrepancies.

Key insights

Property templates enable scalable, accurate dual-track validation of software correctness using formal proofs and property-based testing.

Principles

• Formal proof and PBT are complementary for strong validation.
• Recurring property patterns can be abstracted into templates.
• Templates reduce LLM hallucination and synthesis cost.

Method

The DualVeri framework uses property templates to parameterize shared proof structures for Lean 4 verification and PBT templates for executable PySpark tests, guiding LLMs to fill specific holes.

In practice

• Apply property templates to validate recurring system invariants.
• Use Lean 4 for formal proof and PySpark for PBT.
• Cross-validate proof and test results to find model gaps.

Topics

• Property-Based Testing
• Formal Verification
• Apache Spark
• Lean 4
• AI-Assisted Software Engineering
• Data-Intensive Computing

Code references

• apache/datafusion-comet

Best for: Research Scientist, AI Scientist, Software Engineer, Machine Learning Engineer

Related on AIssential

• Finding Bugs with Claude and Property-based Testing — An AI agent using Claude and property-based testing effectively identifies bugs in major Python libraries by inferring code properties.
• FVSpec: Real-World Property-Based Tests as Lean Challenges — FVSpec benchmarks AI for formal software verification by translating 11,039 Python property-based tests into 9,415 Lean 4 specifications.
• Why AI-Built Apps Pass Every Test and Still Break in Front of Users — AI-built apps fail because agents generate structurally valid but semantically incoherent data, leading to false-positive test results.
• BugBash'26 Morning of Day 1 — Peter Alvaro's 20-year journey in distributed systems reveals no silver bullet, but continuous progress through diverse, evolving approaches.
• This Is The ONLY Way to Trust Your AI Agent — Robust specifications and advanced testing, like property-based testing, are crucial for ensuring AI agent reliability and preventing "wandering" behavior.
• Property-Guided LLM Program Synthesis for Planning — Property-guided LLM program synthesis with counterexamples drastically reduces generation and evaluation costs while improving program quality.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.