The Energy Barrier Reshaping AI Hardware

2026-07-10 · Source: Big Data & AI News - EE Times · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation, Cloud Computing & IT Infrastructure · Depth: Expert, medium

Summary

At Leti Innovation Days 2026, researchers highlighted energy efficiency as the critical constraint reshaping AI hardware roadmaps. Jean-René Lèquepeys of CEA-Leti and Stephan Guttowski from FMD emphasized that scaling computing systems demands rethinking design to avoid unsustainable energy costs. For large AI systems, data movement, including memory access and interconnects, accounts for a significant portion of energy consumption—estimated at 40% and 30% respectively for LLM workloads, with compute at 10%. Solutions involve shortening data paths through technologies like high-bandwidth memory (HBM), 3D integration, near-memory computing, and optical communication, which offers 3-10 picojoules per bit for silicon photonics. Efficient power delivery, particularly the 54V DC to 0.7V DC conversion, requires innovation in wide-bandgap materials. The consensus points to system-technology co-optimization (STCO) across 15 technology areas, integrating processors, memory, interconnects, packaging, power delivery, and cooling to optimize total system energy.

Key takeaway

For AI Hardware Engineers and Architects designing next-generation systems, your focus must shift beyond raw compute power to holistic energy efficiency. Prioritize system-technology co-optimization (STCO) to integrate memory, interconnects, power delivery, and cooling from the outset. This approach is crucial for economic viability and sustainable scaling, as data movement and power conversion losses increasingly constrain performance and drive up operational costs in large AI deployments.

Key insights

AI hardware scaling is now primarily limited by energy consumption, driven by data movement and power delivery.

Principles

• System-level energy efficiency requires more than chip miniaturization.
• Data movement and interconnects are major energy sinks.
• Co-optimize all system components for total energy reduction.

Method

Adopt system-technology co-optimization (STCO) by integrating electrical, thermal, mechanical, and geometric design across 15 technology areas, connecting existing CAD tools via shared data interfaces.

In practice

• Utilize HBM and 3D integration to reduce data travel distance.
• Investigate wide-bandgap materials for power conversion efficiency.
• Design cooling and power delivery as part of chip architecture.

Topics

• AI Hardware
• Energy Efficiency
• System-Technology Co-Optimization
• Data Movement
• 3D Integration
• Power Delivery

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Hardware Engineer, AI Architect, Research Scientist

Related on AIssential

• Accelerating Chipmaking Innovation for the Energy-Efficient AI Era — The AI era demands a coupled, accelerated co-innovation model for energy-efficient semiconductor design, replacing slow, siloed R&D.
• As AI data centers hit power limits, Peak XV backs Indian startup C2i to fix the bottleneck — Efficient power delivery is critical for scaling AI data centers and reducing operational costs.
• CEA-Leti CEO: AI’s Real Bottleneck Is Architecture — AI's future hinges on architectural innovation, integrating memory, compute, and communication efficiently, alongside addressing critical energy constraints.
• Imec’s Patrick Vandenameele: Full-stack Innovation Is the Name of the Game — Future AI progress depends on deep, full-stack co-optimization across compute, memory, photonics, and packaging.
• AI, energy and infrastructure: The future of data centre infrastructure in a high-demand world — Balancing escalating AI-driven compute demand with stringent sustainability goals requires advanced cooling, renewable energy, and intelligent optimization.
• This Will Power Everything — Optical interconnects are critical for scaling AI data centers beyond copper's physical and thermal limitations.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Big Data & AI News - EE Times.