Governance Is Not a Prompt

2026-01-11 · Source: Agus’s Substack · Field: Legal & Regulatory — Compliance & Risk Management, Regulatory Affairs & Government Relations, Artificial Intelligence & Machine Learning · Depth: Advanced, long

Summary

On April 17, 2026, the Federal Reserve, OCC, and FDIC issued SR 26-2, new interagency guidance on model risk management, which notably excludes agentic AI from its scope. This exclusion, replacing the 15-year-old SR 11-7, signals that the existing model risk management paradigm is insufficient for agentic AI, which operates as a system using models to perceive, reason, plan, and act across multiple steps, often autonomously. Current industry practices, such as strong system prompts, flat-file memory, and LLM-based judges, are deemed inadequate for serious governance in high-risk sectors like banking, healthcare, and law. These methods provide "soft influence" rather than the "hard control" required for systems that can make consequential decisions. The regulators have left the governance of agentic AI open for future rulemaking, indicating a need for a new, robust framework.

Key takeaway

For CTOs and VPs of Engineering/Data developing or deploying agentic AI in regulated environments, recognize that current "standard harness" governance approaches are insufficient. You must prioritize building external, auditable governance structures that enforce policies deterministically, rather than relying on probabilistic model interpretations. This shift is critical to avoid regulatory exposure and ensure true accountability, moving beyond mere compliance appearance to substantive control.

Key insights

Existing model risk management frameworks are inadequate for agentic AI, necessitating a new governance paradigm focused on external, enforceable controls.

Principles

• Governance is a structure, not a feeling.
• Separate thinking from deciding in agentic AI.
• Governance must be enforceable, not merely advisory.

Method

A governed agentic architecture requires externalizing policies into structured, executable rules, maintaining authoritative state with provenance, using typed verification, gating actions, and constraining workflows with behavioral contracts.

In practice

• Implement policies as executable rules, not just prose.
• Use deterministic checks for numerical and threshold verification.
• Gate agent actions at the tool layer based on prerequisites and permissions.

Topics

• Agentic AI Governance
• Model Risk Management (SR 26-2)
• Financial Sector Regulation
• External Policy Enforcement
• LLM Judge Limitations

Best for: CTO, VP of Engineering/Data, Executive, Legal Professional, Policy Maker, Director of AI/ML

Related on AIssential

• Drowning In Rules: Navigating America’s AI Regulatory Patchwork — Fragmented US AI regulations create compliance chaos, hindering innovation and increasing litigation risk for businesses.
• The Real Question to Ask About AI Governance — The core problem in AI governance is the lack of clear human authority to stop misbehaving models, not a deficit of monitoring tools.
• Mechanical Enforcement for LLM Governance:Evidence of Governance-Task Decoupling in Financial Decision Systems — External mechanical enforcement significantly improves LLM governance quality and auditability in regulated financial systems.
• HCLTech: How to Ensure AI Compliance and Responsibility — Agentic AI's proactive capabilities demand integrated governance and ethical principles from design to deployment.
• ​Building trustworthy AI: A practical framework for adaptive governance — Modern AI agent governance requires adaptive, risk-based models enforced by platforms, not outdated manual processes.
• Enterprise AI Governance Beyond Model Risk: Why the Control Plane Is Becoming the Real Enterprise… — Modern AI governance must extend beyond models to the entire control plane, ensuring accountability and continuous oversight.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Agus’s Substack.