LLMs as ASP Programmers: Self-Correction Enables Task-Agnostic Nonmonotonic Reasoning

2025-06-11 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

The LLM+ASP framework, developed by Adam Ishay and Joohyung Lee, integrates large language models (LLMs) with Answer Set Programming (ASP) to address challenges in complex reasoning, including high computational costs and logical inconsistencies. Unlike prior neuro-symbolic methods, LLM+ASP is task-agnostic, requiring no per-task engineering or manually authored knowledge modules. It employs an automated self-correction loop, where structured feedback from the ASP solver enables iterative refinement of generated ASP code. Evaluated across six diverse benchmarks, the framework achieved 78.0% average accuracy, a 52.1% relative improvement over baselines. Key findings include ASP's superior performance on nonmonotonic tasks compared to SMT-based alternatives, the self-correction loop as the primary performance driver, and the observation that compact in-context reference guides significantly outperform verbose documentation, revealing a "context rot" phenomenon.

Key takeaway

For AI Scientists and ML Engineers developing reasoning systems, consider integrating LLMs with Answer Set Programming (ASP) to tackle complex, nonmonotonic problems. Your teams should prioritize implementing iterative self-correction loops, as this mechanism is the primary driver of performance gains, effectively replacing extensive manual prompt engineering. Furthermore, when providing in-context learning, opt for compact, focused reference guides to avoid "context rot" and improve LLM adherence to constraints.

Key insights

Automated self-correction with symbolic solver feedback significantly enhances LLM reasoning, especially for nonmonotonic tasks.

Principles

• Nonmonotonic logic (ASP) naturally expresses default rules and exceptions.
• Iterative self-correction replaces handcrafted domain knowledge.
• Compact reference guides improve LLM constraint adherence.

Method

The LLM generates/updates ASP code from natural language, which an ASP solver executes. Structured solver feedback then drives iterative refinement until a correct solution is accepted.

In practice

• Use ASP for problems requiring defeasible reasoning or exceptions.
• Implement self-correction loops with solver feedback for LLM-generated code.
• Provide concise, targeted reference guides over verbose documentation.

Topics

• Neuro-symbolic AI
• Answer Set Programming
• LLM Self-Correction
• Nonmonotonic Reasoning
• Context Rot
• Logical Reasoning Benchmarks

Code references

• azreasoners/llm-asp-programmer
• potassco/guide

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• The Self-Correction Illusion: LLMs Correct Others but Not Themselves — LLMs' self-correction failure is a chat-template artifact, not a cognitive deficit, due to addressability.
• Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs — Code-on-Graph integrates LLMs with KGs using programmatic reasoning, representing schemas as Python classes for flexible, scalable knowledge utilization.
• NEW: From AI Skills to "Skill Programs" — The HASP framework uses deterministic "program functions" to prevent LLM failures by actively intervening in reasoning trajectories.
• Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training — Reasoning LLM-as-judges outperform non-reasoning judges in non-verifiable LLM post-training, despite generating adversarial outputs.
• Recursive Think-Answer Process to Stop "Thinking" Earlier — Integrating confidence-aware self-correction and RL-aware distillation improves LLM reasoning and efficiency.
• How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoning — Benign self-reading patterns in LLMs, marked by focused, forward-shifting attention, correlate with correct quantitative reasoning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.