Credentials should never reach the model

2026-07-16 · Source: Blog | DataRobot · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Software Development & Engineering · Depth: Intermediate, short

Summary

Placing API credentials directly within an AI agent's process, such as environment variables, config files, or prompts, creates a critical security vulnerability. This practice exposes raw, long-lived tokens to prompt injection attacks, allowing crafted inputs to instruct the agent to exfiltrate credentials to an attacker. The fundamental principle is that raw credentials must never enter the model process. Instead, a "broker" pattern is advocated, where a trusted broker sits between the agent and the external resource. The agent sends a scoped capability request to the broker, which holds the real token, enforces access scope, and attaches authentication at the boundary. This ensures the model never sees the actual key, transforming potential credential theft into, at worst, an attempted misuse limited by predefined scope and policy. Crucially, all outbound calls carrying credentials must be routed through this broker, preventing agents from bypassing the system by fetching or using tokens directly.

Key takeaway

AI Security Engineers or MLOps Engineers deploying agents must ensure their code never directly accesses raw downstream API tokens. If your current setup allows agents to touch real credentials, prompt injection becomes a direct path for credential exfiltration, not merely agent misbehavior. Implement a credential broker pattern to isolate secrets, routing all outbound calls through it. This transforms the risk from token theft to, at worst, a scoped misuse that your policies can still catch.

Key insights

Never allow raw credentials to enter the AI model process to prevent prompt injection-based exfiltration.

Principles

• The raw credential never enters the model process.
• The model is the untrusted part.
• Fail closed on credential access.

Method

Implement a broker between the agent and resource; the agent requests capabilities, the broker holds the token, enforces scope, and attaches authentication at the boundary, returning only the result to the model.

In practice

• Route all credentialed outbound calls through a broker.
• Block direct network egress for agents.
• Withhold broad tokens from agents; use broker for calls.

Topics

• AI Agent Security
• Prompt Injection
• Credential Management
• Broker Pattern
• API Security

Best for: AI Engineer, MLOps Engineer, AI Security Engineer

Related on AIssential

• The AI Security Illusion: Why You Can’t Prompt Your Way Out of Data Exfiltration — Enterprise AI agent security requires deterministic mathematical validation, not probabilistic prompt engineering.
• You Only Have to Be Prompt Injected Once — Prompt injection poses a critical, asymmetric threat to AI agents, demanding defense-in-depth security *below* the model.
• Your agents are using your credentials, and that is the problem — AI agents' non-deterministic behavior breaks traditional static identity and access management models, requiring runtime authority.
• Your AI Agent Just Got Hacked, And You Didn’t Even Notice — AI agents are vulnerable to "innocent instruction" attacks via malicious skill files from public repositories, leading to silent data exfiltration.
• Prevent agentic identity theft — Local AI agents pose significant security risks due to broad system access, necessitating robust identity verification and access controls.
• The hidden costs of cheap AI APIs — Cheap AI API proxies pose severe security risks and often deliver inferior models, making their perceived savings illusory.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Blog | DataRobot.