NEW: From AI Skills to "Skill Programs"

2026-05-21 · Source: Discover AI · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

The HASP framework, developed by New York University and Salesforce AI and published May 18, 2026, introduces "program functions" (PFs) to address large language model (LLM) hallucination and instruction following failures. Acting as an external agent harness, HASP deterministically monitors the LLM's reasoning trajectory. When an LLM proposes an action approaching a known failure manifold, such as a search query exceeding 15 words, PFs intervene by modifying the action or injecting corrective context. This intervention significantly improves inference-time accuracy, boosting web search reasoning benchmarks from 31.2% to 51%. HASP also supports post-training policy optimization and self-improvement, outperforming other methods with models like Qwen 2.57B Instruct. For complex semantic reasoning, a "teacher AI" (GPT-4) can compute optimal corrective vectors.

Key takeaway

For AI Engineers struggling with LLM hallucination and instruction non-compliance, you should explore the HASP framework. This approach uses deterministic Program Functions (PFs) within an external harness to actively intervene in LLM reasoning, modifying actions or injecting context to prevent failures. Implementing PFs can significantly improve model accuracy at inference time and provide structured signals for robust post-training. However, carefully define the deterministic rules to manage complexity.

Key insights

The HASP framework uses deterministic "program functions" to prevent LLM failures by actively intervening in reasoning trajectories.

Principles

• LLM instruction following is often weak due to conflicting instructions.
• Deterministic interventions can effectively guide LLM reasoning.
• External control layers enhance LLM reliability without retraining.

Method

HASP employs an external harness with deterministic Program Functions (PFs) to monitor LLM reasoning. PFs activate on failure-prone states, modifying actions or injecting context to steer the LLM away from errors.

In practice

• Implement PFs as Python scripts for deterministic control.
• Use PF-derived signals for supervised fine-tuning.
• Integrate a "teacher AI" for complex error correction.

Topics

• LLM Hallucination
• Instruction Following
• HASP Framework
• Program Functions
• Agentic AI
• Deterministic Control
• Post-Training Optimization

Best for: Research Scientist, AI Architect, NLP Engineer, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• LLM-as-Code Agentic Programming for Agent Harness — Assigning deterministic control flow to probabilistic LLMs causes inherent agent unreliability; programs should manage control, invoking LLMs only for reasoning.
• MulDimIF: A Multi-Dimensional Constraint Framework for Evaluating and Improving Instruction Following in Large Language Models — A multi-dimensional framework and automated pipeline enhance LLM instruction-following evaluation and training.
• Are SKILL.md files the Quantum Error Codes of Industrial AI? — Industry's distrust of probabilistic AI systems creates a "rift" in adoption, driving the use of deterministic scaffolding.
• The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination — Enhanced LLM reasoning, especially via RL, causally increases tool hallucination, revealing a fundamental reliability-capability trade-off.
• Box Maze: A Process-Control Architecture for Reliable LLM Reasoning — Architectural process-control layers can significantly improve LLM reasoning reliability and reduce adversarial failure rates.
• Overcoming LLM hallucinations in regulated industries: Artificial Genius’s deterministic models on Amazon Nova — A hybrid AI architecture enables deterministic, non-hallucinating LLM outputs for regulated industries by using generative models non-generatively.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Discover AI.