Return on Tokens (ROT)

2025-08-26 · Source: Not Boring by Packy McCormick · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Emerging Technologies & Innovation · Depth: Intermediate, extended

Summary

Markie Wagner, in collaboration with the author, introduces the concept of Return on Tokens (ROT) as a critical metric for AI investment, challenging the prevalent "tokenmaxxing" approach. The article argues that maximizing token spend, as seen with companies like Uber burning its 2026 Claude Code budget by April and Amazon shutting down its AI leaderboard, is a "mass delusion" leading to unclear value. ROT, defined as (Value of Output - Cost of Tokens) / Cost of Tokens x 100, emphasizes generating more value while spending less. The proposed solution is to use AI as a "compiler" to generate deterministic code for repetitive, high-accuracy tasks, rather than relying on improvisational "Agents" as a runtime. Poetic, Wagner's company, exemplifies this by learning tacit business rules and converting them into adaptive code, achieving "100x less token usage and nines of accuracy" for clients like AIG, SoFi, and Chime, with AIG CEO Peter Zaffino reporting "99%+ quality outcomes."

Key takeaway

For AI/ML Directors evaluating AI investments, shift your focus from raw token consumption to a Return on Tokens (ROT) framework. Prioritize AI as a "compiler" for deterministic code, ensuring high accuracy and cost efficiency for core business processes. This approach minimizes token usage while maximizing value, transforming your business into evolving software that adapts reliably and predictably, rather than improvising expensively.

Key insights

AI's true value lies in compiling human knowledge into deterministic code for reliable, cost-effective execution, not in wasteful "Agentic" improvisation.

Principles

• "Tokenmaxxing" is a flawed metric; focus on Return on Tokens (ROT).
• AI should compile human knowledge into deterministic code.
• "Doing" work is mostly cheap, deterministic execution.

Method

Learn business processes, including tacit rules, using AI to compile them into adaptive, deterministic code. This code then runs, regenerating and testing itself when conditions change.

In practice

• Implement "routing" to use cheaper models for less complex tasks.
• Prioritize deterministic code over improvisational AI Agents for core work.
• Define clear goals for AI applications to prevent "slop."

Topics

• Return on Tokens
• AI Agents
• Deterministic Code
• AI Cost Optimization
• Enterprise AI Architecture
• Business Process Automation

Best for: VP of Engineering/Data, Executive, AI Engineer, Director of AI/ML, AI Architect, CTO

Related on AIssential

• In Defense of Tokenmaxxing — Experimentation with AI token consumption is crucial for enterprise adaptation to agentic AI, despite potential for misuse.
• ‘Tokenmaxxing’ is making developers less productive than they think — Increased AI code generation correlates with disproportionately higher code churn and revision rates, undermining productivity claims.
• How To Cut Your Token Budget By 80% In 3 Steps — Drastically cut AI token costs by prioritizing local inference, structured agent memory, and workflow reorchestration.
• How to Use AI at an Advanced Level While Minimizing Token Consumption — Advanced AI usage prioritizes "minimum sufficient intelligence" to optimize token consumption and system design, not just cost.
• The Real Lesson from OpenAI’s Top Customers: Tokens Aren’t Spend. They’re Leverage — AI is becoming embedded infrastructure, automating cognition-heavy work across diverse sectors and roles, leveling the competitive landscape.
• Notation Matters: A Benchmark Study of Token-Optimized Formats in Agentic AI Systems — Token-optimized formats like TRON significantly reduce LLM token consumption in agentic systems with acceptable accuracy trade-offs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Not Boring by Packy McCormick.