LLM Cost Tracking Solution: How to Monitor and Control AI Spend in Agentic Systems

2026-05-15 · Source: Comet · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, medium

Summary

Opik offers an LLM cost tracking solution designed to provide granular visibility into token usage and expenditure within complex agentic systems. Unlike traditional prompt engineering, agentic systems involve multiple LLM calls, tool invocations, and retrieval steps for a single user query, making cost estimation challenging. Opik addresses this by integrating LLM cost tracking into its free cloud and open-source versions, allowing teams to monitor spend at micro (span) and macro (trace, project) levels. The platform leverages LLM tracing to capture the full execution path of requests, associating token counts and model identifiers with each LLM call to compute costs. This enables users to identify expensive prompts, inefficient routing, or problematic RAG pipelines, and optimize LLM usage without compromising quality, as demonstrated by Pattern's $60K annual savings.

Key takeaway

For AI Engineers or MLOps teams deploying agentic systems, you should integrate LLM cost tracking as a first-class design parameter. Utilize tools like Opik to gain span-level and trace-level visibility into token consumption, enabling you to identify and address cost outliers in prompts or multi-turn flows, thereby optimizing spend without sacrificing model quality. Start by instrumenting a single workflow to quickly gain actionable insights and control your LLM budget.

Key insights

LLM cost tracking requires granular observability into token usage across complex agentic system execution paths.

Principles

• Treat LLM cost as an observability problem.
• Cost visibility must span micro and macro levels.
• Optimize cost and quality simultaneously.

Method

Implement LLM tracing to capture execution paths, associate token counts and model IDs with each span, then aggregate costs at span, trace, and project levels for detailed analysis and optimization.

In practice

• Instrument key agent entry points with Opik.
• Compare trace-level insights to LLM invoices.
• Refactor prompts, routing, and models based on cost/quality data.

Topics

• LLM Cost Tracking
• Agentic Systems
• LLM Observability
• LLM Tracing
• LLM Evaluation

Code references

• comet-ml/opik

Best for: MLOps Engineer, AI Engineer, Director of AI/ML

Related on AIssential

• You Can’t Monitor an AI Agent Like a Web Service. Here’s What I Track Instead. — AI agent monitoring requires custom metrics beyond web service standards to detect silent, costly quality regressions.
• Companies Are Making Claude and Codex Talk Like Cavemen to Stop AI’s Soaring Costs — Companies are simplifying AI outputs to reduce token costs, a direct response to soaring LLM expenditure.
• From Blind Spots to Observability: Operationalizing LLM Apps with OpenLit — Effective LLM operations require robust observability, cost tracking, and prompt management from the earliest development stages.
• 🥇Top AI Papers of the Week — LLM internal states, external environments, and architectural choices profoundly impact performance, safety, and cost.
• 📈 Why AI bills rise as costs fall — AI costs escalate despite falling token prices due to demand elasticity and agents' hidden, amplified token consumption.
• Announcing the Opik Claude Code Plugin: Automatically Configure Observability for Complex Agentic Systems — Automated observability is crucial for complex, nondeterministic AI agents to ensure performance, reliability, and cost efficiency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Comet.