The Model Called a Tool I Never Gave It

2026-07-27 · Source: LLM on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Advanced, long

Summary

The article, "The Model Called a Tool I Never Gave It," explores architectural shifts when Large Language Models (LLMs) interact with external tools residing on client devices outside server control. Traditionally, tool lists are static or managed by controlled servers. However, in scenarios like a voice-activated device in an external building, the client itself dynamically declares its available functions (manifest) to the backend. This introduces complexities: client capabilities vary by device and can change mid-conversation, and the backend cannot redeploy client firmware. A critical distinction is made: the tool list serves as a hint to the model, not a security gate, as models can invent tool calls. Consequently, the backend must validate client-declared tools and arguments. Managing the agent loop's state during client execution requires choosing between client-held tokens, a durable store, or maintaining a persistent duplex connection, each with distinct infrastructure and robustness implications. This paradigm shifts development from prompt engineering to focusing on trust boundaries, failure isolation, and timeout budgeting.

Key takeaway

For AI Architects designing systems with LLMs interacting with external, client-controlled devices, recognize that the tool manifest is untrusted input, not a security boundary. You must implement robust server-side validation for all client-declared tools and arguments to prevent injection risks. Prioritize using existing duplex connections for agent loop state to simplify infrastructure, but design timeouts meticulously. This approach shifts your focus from prompt engineering to critical trust boundaries and failure isolation.

Key insights

When LLM tools run on uncontrolled client devices, the architecture shifts from prompt engineering to trust and reliability.

Principles

• The tool list is a hint, not a security gate.
• Client-declared capabilities are dynamic and untrusted.
• Server-side validation of client-supplied arguments is essential.

Method

The article describes three methods for managing agent loop state: client-held tokens, a durable store, or maintaining a persistent duplex connection.

In practice

• Validate all client-supplied arguments to server tools.
• Design timeouts carefully for client-blocking operations.
• Use existing duplex connections for state management.

Topics

• Large Language Models
• Tool Use
• Client-Server Architecture
• Distributed Systems
• Trust Boundaries
• Agent Loops

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Engineer, Machine Learning Engineer, AI Architect

Related on AIssential

• AI Security & the LLM Threat Model: Why You Can’t Patch Prompt Injection — LLMs' inability to differentiate instructions from data is the root of most LLM security risks, making prompt injection an unsolved problem.
• Tool Forge: A Validation-Carrying Toolchain for Governed Agentic Execution — Tool Forge provides a governed, validated toolchain and token-efficient routing for LLM agents, improving reliability and context management.
• Connecting LLMs to the Real World: Tool Use, Function Calling, and MCP — LLMs use an application layer and standardized protocols to interact with external tools, extending their capabilities beyond text generation.
• Trustworthy Agent Network: Trust in Agent Networks Must Be Baked In, Not Bolted On — Trust in LLM-based agent networks must be architected into the system's core design, not retrofitted.
• SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents — Tool-using LLM agent security requires distinguishing semantic compliance from observable, executable harm.
• The Model May Propose. The Governed System Must Dispose. — LLMs should propose actions, but a governed system must dispose (decide, authorize, execute) to ensure reliability and safety.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by LLM on Medium.