MaxSAT Fuzzing and Delta Debugging

2026-02-09 · Source: Journal of Artificial Intelligence Research · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, quick

Summary

A systematic study evaluated automated fuzzing techniques for Maximum Satisfiability (MaxSAT) solvers, combining large-scale stress testing with a novel MaxSAT-specific delta debugging method. This approach generated millions of structured MaxSAT instances and efficiently isolated failure-inducing inputs, distilling them into minimal counterexamples. Over 100 hours of fuzzing, previously unknown failures were discovered in almost all 43 solvers from recent MaxSAT competitions and in three certificate-producing solvers. These failures included crashes, incorrect optimality bounds, and severe performance slowdowns, with one certified solver exhibiting a critical soundness error. The resulting corpus of minimal counterexamples was published as a public regression suite, which was adopted for the 2024 MaxSAT Evaluation, reducing average solver failure rates by over 50%.

Key takeaway

For AI scientists developing or deploying MaxSAT solvers, this study highlights the critical need for robust validation. You should integrate systematic fuzz testing and delta debugging into your development pipeline to preemptively identify and fix soundness errors, crashes, and performance issues. Adopting the published regression suite (Paxian 2025a) can immediately improve solver reliability and ensure compliance with community standards like the 2024 MaxSAT Evaluation.

Key insights

Systematic fuzzing and delta debugging significantly enhance MaxSAT solver reliability and development.

Principles

• Combine stress testing with targeted debugging.
• Isolate and minimize failure-inducing inputs.
• Publish counterexamples for community adoption.

Method

Orchestrate parallel generation of millions of structured MaxSAT instances, then apply MaxSAT-specific delta debugging to isolate and distill failing cases into minimal counterexamples for precise failure localization.

In practice

• Integrate fuzzers into solver development workflows.
• Use minimal counterexamples for regression testing.

Topics

• MaxSAT Solvers
• Fuzz Testing
• Delta Debugging
• Solver Robustness
• Regression Testing

Best for: AI Scientist, AI Researcher, Research Scientist, Software Engineer

Related on AIssential

• FuzzingRL: Reinforcement Fuzz-Testing for Revealing VLM Failures — FuzzingRL automatically generates adversarial questions to expose and amplify Vision-Language Model vulnerabilities through iterative refinement.
• Code-Augur: Agentic Vulnerability Detection via Specification Inference — Agentic vulnerability detection improves by making LLM assumptions explicit and continuously validating them with fuzzing.
• A 91% eval pass rate shipped our worst regression. We gate on the delta now. — Absolute evaluation thresholds are insufficient for regression detection; delta-based gating against a stable baseline is crucial.
• Do a sanity check on your experiments — Manual sanity checks on data, model outputs, and evaluations are crucial for detecting common experimental bugs.
• PBFuzz: Agentic Directed Fuzzing for PoV Generation — PBFuzz leverages agentic LLMs and property-based testing to bridge semantic understanding and efficient PoV input generation.
• Cut Your Debugging Time By 80% with AI Skills And This Workflow — Combining TDD, vertical slicing, and AI tools significantly boosts code quality and reduces debugging.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Journal of Artificial Intelligence Research.