If an LLM could genuinely deduce the intellectual baseline of its user from provided documents or conversational context, it could eliminate boilerplate introductions, redundant explanations, and...

2025-11-28 · Source: Pascal’s Substack · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

Current Large Language Models (LLMs) fundamentally lack the cognitive capacity for genuine epistemic modeling, meaning they cannot accurately deduce a user's prior knowledge from textual inputs. Instead, models rely on "Statistical Quasi-Equivalence" and linguistic approximations, leading to "Verbosity Compensation" (VC) and a systematic violation of Grice's Maxim of Quantity. This inherent verbosity is exacerbated by Reinforcement Learning from Human Feedback (RLHF) training, which exhibits a "length bias" from human annotators. VC manifests in various forms, including excessive enumeration and redundant explanations, and significantly degrades factual accuracy, with recall drops up to 24.72% on datasets like Qasper. Economically, this creates a "verbosity tax" because output tokens cost 4x to 6x more than input tokens. For example, reducing output from 1,500 to 300 tokens can yield an 80% cost reduction. This cost is further magnified in agentic frameworks (1.3x to 5x multipliers) and reasoning models. Mitigations include Frictive Policy Optimization (FPO), Cost-Regularized Optimization of Prompts (CROP), and semantic caching.

Key takeaway

For MLOps Engineers optimizing LLM deployments, your current models are likely incurring a significant "verbosity tax" due to their inability to gauge user expertise. You should prioritize implementing algorithmic restraints like Frictive Policy Optimization (FPO) or Cost-Regularized Optimization of Prompts (CROP) to enforce conciseness. Additionally, deploy semantic caching to reduce redundant output token generation, which can cut operational costs by up to 80% and improve user experience.

Key insights

LLMs' lack of true epistemic modeling drives costly verbosity and reduces accuracy, necessitating explicit mitigation strategies.

Principles

• LLMs use statistical shortcuts, not true Theory of Mind.
• RLHF training introduces a pervasive length bias.
• Output tokens are 4x-6x more expensive than input tokens.

Method

Frictive Policy Optimization (FPO) formalizes alignment as a risk-sensitive epistemic control problem, using clarifying questions to mitigate verbosity and hallucination. CROP also penalizes token bloat.

In practice

• Implement semantic caching to bypass LLM generation for repeated queries.
• Apply CROP to balance logical correctness with generative brevity.
• Use RLPA to dynamically refine user profiles and adapt response complexity.

Topics

• Large Language Models
• Epistemic Modeling
• Theory of Mind
• Token Economics
• Reinforcement Learning from Human Feedback
• Frictive Policy Optimization
• Semantic Caching

Best for: AI Engineer, Research Scientist, Entrepreneur, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Disable “Thinking,” Still Get Thousands of Tokens: What Instruct LLMs Are Doing — Many "Instruct" LLMs implicitly use significant reasoning budgets, skewing benchmark results and increasing inference costs.
• TLPO: Token-Level Policy Optimization for Mitigating Language Confusion in Large Language Models — TLPO uses token-level policy optimization to mitigate language confusion in LLMs without degrading general capabilities.
• The Context Problem — AI's "context" as a billing unit conflates capacity with coherence, leading to inflated costs and degraded model performance.
• Human Mind Reconstruction w/ AI (ToM, UserHarness) — Decoupling epistemic structure from LLM generation significantly boosts reasoning accuracy and efficiency, even for smaller models.
• Taught Claude to talk like a caveman to use 75% less tokens. — Prompting AI models for extremely concise, "caveman"-style responses may reduce token usage but risks quality.
• Finding the Optimal Reasoning Budget for LLMs — Optimal LLM reasoning length varies by model, while context compression and reasoning failures are critical research areas.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Pascal’s Substack.