I Built a Kill Switch for AI Agent Tool Calls — and Used SigNoz to Prove It Worked

2026-07-26 · Source: Artificial Intelligence on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Software Development & Engineering · Depth: Intermediate, medium

Summary

TRACEBACK is a system designed to implement a "kill switch" for AI agent tool calls, preventing syntactically valid but unsafe actions. It addresses the problem where an AI agent might propose a dangerous action, like refund_tool(order_id="1234", amount="free"), which is valid API-wise but violates business rules. TRACEBACK separates responsibilities: the agent proposes, a deterministic policy gate decides whether to allow the action, and OpenTelemetry with SigNoz records evidence. The system demonstrates two runs: an unsafe baseline where the tool is attempted, and a guarded replay where an in-process TypeScript policy gate blocks the action, evidenced by the absence of a tool.refund_tool span in SigNoz traces. It uses traces, logs, and metrics for comprehensive observability, including a traceback.policy.blocked_refunds metric.

Key takeaway

For AI Engineers building agentic systems, you must implement a robust, deterministic policy gate as the security boundary for tool calls. Do not rely solely on prompt engineering for safety; instead, let your model propose actions, but ensure deterministic code explicitly decides execution. Use comprehensive observability, like OpenTelemetry and SigNoz, to verify that unsafe actions are blocked before reaching any external tool, providing auditable proof of prevention.

Key insights

A deterministic policy gate, not the AI model, must serve as the security boundary for agent tool calls.

Principles

• Validate actions before side effects.
• A proposal is not permission.
• Observability proves prevention, not just logging.

Method

Implement a deterministic, in-process policy gate to evaluate agent proposals against business rules, emitting OpenTelemetry signals (traces, logs, metrics) to an observability platform like SigNoz to prove execution flow and blocking decisions.

In practice

• Use OpenTelemetry to trace agent proposals and policy gate decisions.
• Configure a policy gate to check instruction patterns and proposed amounts against stored data.
• Monitor policy.gate.evaluate error statuses and missing tool spans for proof of blocking.

Topics

• AI Agents
• Tool Calls
• Policy Gates
• OpenTelemetry
• SigNoz
• AI Safety

Best for: AI Engineer, MLOps Engineer, AI Security Engineer

Related on AIssential

• AgentGuard: Putting a Firewall in Front of What AI Agents Actually Do — AgentGuard verifies AI agent actions pre-execution using a layered defense, enhancing safety beyond prompt-based guardrails.
• # Building Tower Trace: stopping agent tool calls and proving it in SigNoz — Agent tool calls require explicit control plane enforcement, not just prompt-based safety, with observability for verification.
• Claude Code, Codex and Agentic Coding #8 — AI coding agents are rapidly advancing but require careful management to mitigate risks from autonomous actions.
• Reliable AI Agents Need a Smaller Decision Surface — Reliable AI agents limit model autonomy to ambiguous decisions, delegating deterministic tasks to conventional software.
• Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes, AWS — AI agent reliability and efficiency are significantly improved by implementing code-level controls rather than relying solely on prompt engineering.
• Engineering an Autonomous AI Study Agent: Tracing the Agentic State Machine — Robust AI agents require deep, full-stack observability via OpenTelemetry and unified platforms like SigNoz to ensure reliability and control.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence on Medium.