I Ran a Bank's Risk Model On An AI Agent

2026-07-16 · Source: The AI Agent Architect · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Intermediate, medium

Summary

An analysis of an AI agent deployed at a major logistics operator revealed that conversational agents incur unforeseen, compounding costs, particularly during high-stress events. A warehouse-incident support agent, built on Claude Sonnet, consumed 80,000 input tokens for every 1,200 produced, with over 90% of the bill attributed to re-sending accumulated memory with each turn. This architecture means costs escalate significantly during multi-turn crisis conversations, making agents most expensive when the business is already struggling. To address this, the author applied a banking-style Value-at-Risk model, creating scenario tables for normal, elevated, critical, and extreme months (weighted 85%, 10%, 4%, 1%). This revealed critical months could cost ten times the base, and the P95 budget figure was five times the expected annual cost, enabling informed rollout decisions.

Key takeaway

For AI Engineers or Directors of AI/ML deploying conversational agents, you must move beyond flat-rate cost projections. Your budget should reflect a price distribution, not a single number, by modeling worst-case scenarios. Analyze token logs to identify compounding architectures where input tokens dominate. Present ROI as a confidence range (e.g., 70%, 50%, 30%) to accurately reflect production realities and avoid mid-crisis financial surprises.

Key insights

AI agent costs compound with conversation length due to memory re-sending, requiring distribution-based pricing.

Principles

• Agent costs compound, not grow linearly.
• Demo costs (P50) don't reflect production (P95).
• Price AI agents as distributions.

Method

Apply a Value-at-Risk (VaR) model by creating scenario tables with weighted probabilities (e.g., 85%, 10%, 4%, 1%) to calculate the P95 budget.

In practice

• Analyze token logs for input/output ratio.
• Present ROI as a range (70%, 50%, 30% confidence).

Topics

• AI Agents
• LLM Costs
• Token Economics
• Value-at-Risk
• Financial Modeling
• Production Deployment

Best for: AI Engineer, MLOps Engineer, Director of AI/ML

Related on AIssential

• You NEED to do this (HUGE AI SAVINGS) — AI cost and quality depend on token "intelligence density" and strategic multi-model workflows, not just per-token price.
• AI's Economics Don't Make Sense — Generative AI's subscription models and compute infrastructure are economically unsustainable due to massive subsidies and hidden true costs.
• 7 AI Agent Cost Optimization Strategies That Cut LLM Bills by Up to 90% — AI agent costs are driven by post-model invocation processes, not just falling token prices, leading to budget overruns despite cheaper LLMs.
• How to Cost an AI Agent Before You Build It — Accurately costing AI agents requires a dedicated ROI framework beyond mere prioritization to ensure financial viability.
• ‘Tokenmaxxing’ Starts to Fade as Companies Eye Agentic Coding Costs — The rapid adoption of AI coding agents is leading to unexpected budget overruns, prompting a re-evaluation of their cost-effectiveness.
• The Cost Conversation Nobody Wants to Have — AI project production failures stem from unaddressed cost conversations, not technical feasibility, due to underestimation of scale factors.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The AI Agent Architect.