What if building more grid capacity isn’t the answer? Solving ‘phantom compute’ could address data centre efficiency

2026-06-03 · Source: Tech Monitor · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Operations & Process Management · Depth: Intermediate, short

Summary

Hitachi Vantara's Simon Ninan highlights "phantom compute" as a significant bottleneck in AI development, arguing that inefficient data management, not just grid capacity, hinders data center efficiency. He defines "phantom compute" as allocated GPU capacity that is idle or performing useless work due to vast amounts of "junk data" accumulated in unstructured data lakes. Ninan contends that optimized data processes could increase GPU efficiency by 30% and reduce compute demand by 30-40% without impacting AI processing speed. With hyperscalers projected to spend nearly \$685bn in combined capital expenditure by 2026 on AI infrastructure, he suggests that an AI-driven Data Center Infrastructure Management (DCIM) solution, coupled with robust data governance, could boost compute capacity from 40-60% and yield substantial savings by cascading positive effects on cooling and overall power consumption.

Key takeaway

For AI Architects and MLOps Engineers planning new data center infrastructure or optimizing existing deployments, recognize that "phantom compute" from poor data governance is a critical bottleneck. Prioritize robust data management strategies and AI-driven DCIM solutions to significantly improve GPU utilization by 30-80% and reduce overall compute demand by 30-40%. This approach directly impacts AI project ROI and yields substantial operational savings, rather than solely focusing on grid capacity expansion.

Key insights

"Phantom compute" from junk data bottlenecks AI development; efficient data governance can boost GPU utilization and cut compute demand.

Principles

• Data governance drives GPU utilization from 30% to 80%.
• Inefficient data management creates "phantom compute."
• AI ROI requires solving data fundamentals first.

Method

Implement AI-driven Data Center Infrastructure Management (DCIM) solutions to integrate power, cooling, and IT rack management, optimizing data center operations and resource allocation.

In practice

• Scrutinize data management within data centers.
• Disperse demand to the edge for offloading.
• Prioritize clean, reliable training data for AI.

Topics

• Data Governance
• Data Center Efficiency
• Phantom Compute
• AI Infrastructure
• GPU Utilization
• DCIM Solutions

Best for: CTO, VP of Engineering/Data, Executive, AI Architect, MLOps Engineer, Director of AI/ML

Related on AIssential

• We're rapidly expanding our AI infrastructure and have adopted a novel approach of building weather-proof tents to house GPU clusters. This enables us to get new data centers online in months instead of years. 🚀 Read more in this @FastCompany article: fas - x.com — Meta accelerates AI infrastructure deployment by housing GPU clusters in weather-proof tents, reducing build times from years to months.
• 400 Million Idle Gaming PCs Could Be The AI Era's Secret Weapon — Idle gaming PCs offer a massive, distributed compute resource for democratizing AI development and inference.
• Billions in GPU Debt Sounds Insane. Here’s Why Its Not. — Innovative financing structures are critical for scaling capital-intensive AI infrastructure, prioritizing contracted cash flows over hardware collateral.
• AI's GPU problem is actually a data delivery problem — Inefficient data delivery, not GPUs, is the primary bottleneck for AI workload performance and scalability.
• Most AI teams treat compute as a commodity. It's not. — AI compute quality, not just quantity, dictates project success, efficiency, and unit economics for frontier models.
• No Dumb Questions: What is cloud computing and why is everyone doing it? — Cloud computing offers flexibility and scalability by abstracting hardware management to third-party providers.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Tech Monitor.