Why The CPU Is Back

2026-05-08 · Source: The Business Engineer · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Advanced, quick

Summary

Arm recently achieved a record quarter, doubling its AGI CPU demand in six weeks, driven by a shift in the AI scaling curve. The AI landscape is characterized by three distinct regimes, each moving the computational bottleneck to a different silicon layer. The first two regimes, focused on pretraining and inference-time scaling, primarily benefited NVIDIA. However, the current third regime, centered on agentic scaling, has shifted demand towards CPUs, specifically benefiting Arm. This shift does not come at NVIDIA's expense but rather represents an additional layer of compute consumption, indicating a broader expansion of total compute demand across different hardware types.

Key takeaway

For VPs of Engineering and Data evaluating future AI infrastructure investments, recognize that the shift to agentic scaling fundamentally alters hardware requirements. Your strategy should now account for increased CPU demand, particularly Arm-based solutions, alongside existing GPU infrastructure to optimize for emerging AI workloads and maintain competitive performance.

Key insights

AI scaling has shifted to agentic workloads, driving significant CPU demand for Arm.

Principles

• AI scaling involves three distinct regimes.
• Each regime redistributes compute consumption.
• Agentic scaling drives CPU demand.

In practice

• Monitor agentic AI workload growth.
• Evaluate CPU architectures for AI inference.

Topics

• Arm CPU Demand
• AI Scaling Regimes
• Agentic Scaling
• Pretraining Scaling
• Inference-Time Scaling

Best for: Investor, VP of Engineering/Data, MLOps Engineer, Director of AI/ML, AI Architect, CTO

Related on AIssential

• TAI #197: Anthropic Turned the OpenClaw Demand Signal Into a Product — Agentic AI advancements are rapidly increasing compute demand, straining a constrained global AI supply chain.
• Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective — CPU-centric bottlenecks in agentic AI workloads significantly impact latency, throughput, and energy consumption, often more than GPU inference.
• 🎯 Magnitudes of intelligence — Increasing AI token consumption by orders of magnitude fundamentally changes how AI is used and integrated into workflows.
• The Agentic Expansion Cascade — The agentic expansion is a substrate-driven consequence of parallel AI scaling laws, not a product strategy.
• Marc Andreessen introspects on The Death of the Browser, Pi + OpenClaw, and Why "This Time Is Different" — Current AI advancements are an "80-year overnight success" built on foundational research, now demonstrating real-world functionality.
• 😸 You're behind on AI. Here's the recap — The AI landscape is rapidly advancing across agents, benchmarks, and hardware, driving significant shifts in market dynamics.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Business Engineer.