Chips Topic Overview
Summary
AI progress is heavily reliant on specialized AI chips, or accelerators, manufactured predominantly by Taiwan Semiconductor Manufacturing Company (TSMC). Key examples include Nvidia's Blackwell and Hopper GPUs, Google's TPU, and Amazon's Trainium series. These chips are crucial for training and deploying frontier AI models, leading to demand exceeding supply. The complex supply chain involves designers like Nvidia, Google, and AMD, memory providers (Samsung, SK Hynix, Micron), and critical equipment suppliers like ASML for EUV machines. Despite rising individual chip prices (e.g., Nvidia H100 at \$34,000 in 2022), cost-effectiveness, measured as compute per dollar, has doubled every 2.5 years due to architectural improvements and increased transistor density. While individual chip energy efficiency improves by 40% annually, total AI electricity consumption is rising, reaching "tens of gigawatts" by late 2025, comparable to New York's peak demand, as chip installations outpace efficiency gains.
Key takeaway
For AI Architects and Directors of AI/ML planning future infrastructure, recognize that while individual AI chip prices are rising, their cost-effectiveness per unit of computation continues to improve significantly, doubling every 2.5 years. However, your total electricity consumption will still increase as chip installations outpace efficiency gains, potentially straining power grids. Prioritize supply chain diversification where possible and factor in substantial power infrastructure upgrades for scaling AI operations, as the global supply chain remains highly concentrated and vulnerable to single points of failure like TSMC and ASML.
Key insights
Global AI progress hinges on a concentrated supply chain for specialized chips, driving both innovation and infrastructure challenges.
Principles
- AI chip supply chain is highly concentrated.
- Compute per dollar improves with each generation.
- Total AI power consumption is increasing.
In practice
- Evaluate AI chip cost by compute per dollar.
- Monitor HBM supply as a key constraint.
- Factor in rising data center power demands.
Topics
- AI Chips
- Semiconductor Supply Chain
- NVIDIA Blackwell
- TSMC
- AI Data Centers
- Compute Cost-Effectiveness
- Energy Efficiency
Best for: Investor, CTO, VP of Engineering/Data, AI Hardware Engineer, AI Architect, Director of AI/ML
Related on AIssential
See Counsel's argued verdicts on the open AI decisions leaders are weighing →
Editorial summary, takeaway, and curation by AIssential. Original article published by Papers & Reports | Epoch AI.