The Epoch Brief - May 22, 2026

2026-01-28 · Source: Epoch AI · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Emerging Technologies & Innovation · Depth: Intermediate, quick

Summary

Epoch AI's latest Data Insight reveals that high-bandwidth memory (HBM) has become the largest and fastest-growing component cost for leading AI chip designers, increasing from 52% to 63% of total component spending between Q1 2024 and Q4 2025. This surge saw HBM expenditure across chips from Nvidia, AMD, Google, and Amazon escalate from approximately \$12 billion in 2024 to \$32 billion in 2025. Concurrently, a Gradient Update by Josh You estimates that while top frontier labs currently use less than half of global AI compute, they could consume most available capacity within a few years. This would cap further scaling by chip supply, necessitating a dramatic acceleration in compute production, despite AI capital expenditure already nearing \$1 trillion annually. Additionally, FrontierMath: Open Problems workshops are commencing May 26 to identify unsolved research math problems.

Key takeaway

For Directors of AI/ML planning future infrastructure, recognize that high-bandwidth memory (HBM) costs are escalating rapidly, now comprising nearly two-thirds of AI chip expenses. You should factor this into your procurement strategies and budget forecasts. Additionally, understand that frontier lab compute usage is approaching global supply limits, indicating that future scaling will demand significant capital expenditure increases or innovative efficiency gains to avoid bottlenecks.

Key insights

AI's scaling trajectory faces dual pressures: rapidly increasing HBM costs and impending compute supply limits for frontier labs.

Principles

• Memory is the dominant and fastest-growing AI chip cost.
• Frontier AI compute growth is nearing supply constraints.

In practice

• Monitor HBM cost trends for AI hardware procurement.
• Assess future compute needs against global supply projections.

Topics

• AI Chip Costs
• High-Bandwidth Memory
• AI Compute Scaling
• Frontier AI Labs
• AI Capital Expenditure
• AI Infrastructure

Best for: CTO, VP of Engineering/Data, Executive, AI Scientist, Director of AI/ML, Investor

Related on AIssential

• The AI Memory Tax & the Bifurcation of AI Scaling Laws — Memory has transitioned from a commodity to a strategic asset due to AI's high bandwidth demands.
• AI chips are pushing everything else off TSMC's most advanced production lines — Simultaneous AI accelerator migration to TSMC's N3 process is creating a severe, multi-year capacity crunch.
• SK Hynix, Micron Join $1 Trillion Market Cap Club — The AI boom is driving unprecedented demand and valuation for HBM chipmakers and AI software, while also raising geopolitical and economic questions.
• ChatGPT Designs Cancer Vaccine — Semiconductor manufacturing, particularly EUV lithography, is the ultimate bottleneck for AI compute scaling, not power or data centers.
• RAM Fever — AI server demand is causing significant DRAM price surges, granting memory makers substantial pricing power.
• Broadcom’s AI Chip Guidance Raises APAC Supply-Chain Questions — Broadcom's AI chip guidance highlights supply chain volatility and margin scrutiny for HBM, not a cooling of AI demand.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Epoch AI.