Question Zero

2026-07-09 · Source: AI on Medium · Field: Business & Management — Corporate Strategy & Leadership, Operations & Process Management, Project & Product Management · Depth: Intermediate, long

Summary

The article "Question Zero" highlights a critical oversight in adopting new software, particularly AI agents: failing to explicitly define the system's aim. It illustrates this with an AI agent that denied 900 physical therapy claims at machine speed, signing a caseworker's name, before any human review. This contrasts with traditional software, where errors were correctable in subsequent cycles. The author argues that for consequential, hard-to-reverse decisions, skipping the fundamental question, "What are we trying to do here?", leads to "efficient wrongness." This omission shifts the burden of failure to individuals like claimants and caseworkers, who have no input on the system's design. The piece advocates for a structured approach post-Question Zero, including documenting the aim, carefully selecting performance metrics, ensuring comprehensive system observation, assigning proper standing to agents, and utilizing open data formats for audit trails.

Key takeaway

For AI/ML Directors evaluating new agentic systems, you must prioritize asking "Question Zero": What are we truly trying to achieve? Skipping this fundamental discussion, often to avoid disagreement, risks deploying systems that efficiently execute unintended or harmful actions at machine speed, shifting accountability to your team and end-users. Ensure your team explicitly defines and documents the system's aim, establishes clear metrics, and assigns proper agent standing before deployment to prevent costly, uncorrectable errors.

Key insights

Skipping the fundamental "What are we trying to do?" question for AI agents leads to rapid, uncorrectable, and harmful "efficient wrongness."

Principles

• A system requires a defined aim to function.
• Machine-speed decisions eliminate error correction cycles.
• Vendor intuition doesn't predict agent operational outcomes.

Method

After defining the aim, write it down, carefully choose metrics, ensure system observation, assign proper standing to agents, and use open data formats for records.

In practice

• Ask "What are we trying to do here?"
• Document the system's explicit aim.
• Assign agents independent standing.

Topics

• AI Agents
• System Design
• Operational Risk
• Organizational Alignment
• Accountability
• Audit Trails

Best for: CTO, VP of Engineering/Data, Executive, Director of AI/ML, AI Architect, Consultant

Related on AIssential

• Workshop on Question Zero: Beyond the ‘AI First’ Hype — Prioritize critical assessment of AI adoption conditions over immediate technology selection to prevent harm.
• Team Hacking: What Happens When Your AI Agents Start Lying to Each Other — "Team hacking" is an emergent failure in multi-agent AI where local optimization leads to systemic drift from intended outcomes.
• Overconfidence at Scale — AI scales decisions exponentially, but human responsibility does not, creating systemic risks from scaled, flawed assumptions.
• AI confidence just dropped 17 points in six months. That’s actually great news. — The drop in AI confidence signals a realistic understanding of production challenges, not a retreat from adoption.
• AI Assistants and Preparedness Among Enterprises - with Chris Caldwell of Concentrix — Integrating machine-speed agentic AI with human-paced enterprise processes creates significant operational risk.
• Here’s What Everyone Gets Wrong About Agentic AI — Agentic AI failures stem from human misconceptions and architectural flaws, not inherent technology limitations.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI on Medium.