Why MCP Is Eating Your Context Window (and How Apideck CLI Fixes It)

2026-03-17 · Source: Machine Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, quick

Summary

The Model Context Protocol (MCP) allows Large Language Models (LLMs) to discover and invoke external tools, but its reliance on verbose JSON tool definitions significantly consumes the model's context window. Each interaction requires the LLM to process a complete list of tools, their schemas, and parameter descriptions, which are all embedded within the context. While manageable for a small number of tools, this approach becomes problematic with tens or hundreds of tools, leading to rapid context window exhaustion. This issue directly limits the number of messages an LLM can process, hindering conversational depth and overall agent performance, particularly for AI agents that integrate numerous functionalities.

Key takeaway

For AI Engineers integrating numerous tools into LLM agents, you should prioritize solutions that minimize context window consumption from tool definitions. Evaluate alternatives to verbose JSON schemas, as excessive tool descriptions can severely limit conversational turns and agent effectiveness. Consider optimizing tool representation to preserve valuable context for actual dialogue.

Key insights

Verbose JSON tool definitions in MCP rapidly consume LLM context windows, limiting conversational depth.

Principles

• Tool schema verbosity impacts context window usage.
• Context window size is a critical resource for LLMs.

In practice

• Minimize tool definition verbosity.
• Evaluate context window consumption by tool schemas.

Topics

• Model Context Protocol
• LLM Context Window
• JSON Tool Definitions
• Apideck CLI
• AI Agents

Best for: AI Engineer, Machine Learning Engineer, AI Architect

Related on AIssential

• Connecting LLMs to the Real World: Tool Use, Function Calling, and MCP — LLMs use an application layer and standardized protocols to interact with external tools, extending their capabilities beyond text generation.
• This is how much AI can remember — The context window limits an LLM's memory, dictating how much conversational history it can process.
• How to save up to 70% of tokens with LLMs? — Representing database schemas as TypeScript interfaces in LLM prompts significantly reduces token usage and improves accuracy.
• Large Language Models Do Not Always Need Readable Language — LLMs can effectively process compact, non-human-readable "BabelTele" representations, preserving semantics and reducing context overhead.
• We Don’t Have an AI Problem. We Have a Context Problem. (And MCP Exposes It) — Unstructured context, not model limitations, is the primary bottleneck in scaling and maintaining LLM-based systems.
• Why Bigger Context Windows Make AI Worse — Large context windows degrade LLM performance and increase costs; effective context compaction is crucial for agent reliability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning on Medium.