Agentic AI strains legacy IT systems

2026-07-10 · Source: Information and Enterprise Technology News | CIO Dive - Www.ciodive.com · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Fundamental Awareness, quick

Summary

Google's 2026 State of AI Infrastructure report, published July 10, 2026, reveals that agentic AI workloads are severely straining legacy IT systems, necessitating significant upgrades for over 83% of organizations. The report, based on a survey of more than 1,400 senior IT leaders, indicates that only 17% have full confidence in their current tech stacks to support mission-critical AI agents. Agentic AI's demand for hundreds of actions per prompt drives high inference costs, now accounting for nearly half of all AI workloads, up from 28% for training. Nirav Mehta, VP of product management at Google Cloud, notes that 62% of IT leaders face high inference costs from data egress, storage bloat, and idle specialized hardware. The report concludes that agentic AI requires a new infrastructure standard, with governance and hybrid multicloud architecture being critical for scaling and addressing digital sovereignty, as 75% of non-U.S. enterprises are expected to adopt sovereignty strategies by 2030. Cost efficiency (96% of leaders) and power consumption (91% of leaders) are paramount in hardware selection.

Key takeaway

For CTOs and AI Architects planning agentic AI deployments, you must prioritize significant infrastructure upgrades. Your legacy systems will likely incur unsustainable inference costs and performance issues, as 83% of organizations face this challenge. Focus on hybrid multicloud architectures and robust governance to manage costs and ensure digital sovereignty. Evaluate hardware based on cost efficiency and power consumption to mitigate rising operational expenses.

Key insights

Agentic AI workloads demand new infrastructure standards due to unsustainable costs and performance strains on legacy IT.

Principles

• Agentic AI drives inference costs higher than generative AI.
• Cost efficiency and power consumption are critical infrastructure factors.
• Hybrid multicloud architecture is key for AI deployment and sovereignty.

In practice

• Evaluate current infrastructure for agentic AI readiness.
• Prioritize cost efficiency in AI hardware selection.
• Consider hybrid multicloud for AI deployment.

Topics

• Agentic AI
• AI Infrastructure
• IT Modernization
• Inference Costs
• Hybrid Multicloud
• Digital Sovereignty

Best for: VP of Engineering/Data, Director of AI/ML, Executive, CTO, AI Architect, IT Professional

Related on AIssential

• AI’s easy on-ramp has become a costly exit problem for enterprises, says Red Hat — Enterprises must transition from expensive frontier AI models to cost-efficient, self-managed inference on horizontal cloud platforms.
• How agentic AI helps heal the systems we can’t replace — Agentic AI can modernize critical legacy systems by learning their real-world behaviors and acting as a stable, synthetic API.
• Dell’s Microsoft/AMD collaboration: Three insights you may have missed from theCUBE’s coverage of Dell Technologies World — AI cost and architectural demands drive strategic partnerships for hybrid, optimized infrastructure solutions.
• The Inevitable Future Of AI And Why Hosted AI is Probably Doomed — AI deployment is evolving from costly cloud-hosted proprietary models to efficient local open-weight solutions.
• Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO — AI agents necessitate a programmatic, elastic infrastructure with rapid feedback loops, diverging from traditional human-centric cloud development.
• Your AI agents will run everywhere. Is your architecture ready for that? — Agentic AI requires infrastructure-agnostic deployment via a vendor-neutral control plane for consistent, secure, and governed operation across diverse environments.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Information and Enterprise Technology News | CIO Dive - Www.ciodive.com.