Agents think in milliseconds, legacy infrastructure doesn't. LinkedIn, Walmart and Zendesk shared how they closed the gap at VB Transform 2026

2026-07-17 · Source: VentureBeat · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Data Science & Analytics · Depth: Advanced, short

Summary

Infrastructure leaders from LinkedIn, Walmart, and Zendesk at VB Transform 2026 concluded that legacy infrastructure, not AI models, is the primary bottleneck slowing down AI agents in production. Enterprise systems, designed for human workflows, struggle with agent speeds. LinkedIn faced issues with Kubernetes' on-demand container provisioning, which was too slow, and structural hallucination when LLMs evaluated other LLMs. They resolved this with pre-provisioned container pools and a scripted control flow, dedicating 80% of workflows to deterministic code. Walmart encountered duplication from "citizen developers" building numerous overlapping agents; this was addressed by implementing governance to identify and promote optimal solutions. Zendesk's challenge stemmed from data infrastructure limitations, specifically handling 20 billion customer conversations. This necessitated investment in robust data pipelines rather than relying solely on large context windows.

Key takeaway

For AI Architects and MLOps Engineers deploying agentic systems, recognize that infrastructure is your primary scaling challenge, not model performance. You should proactively invest in robust evaluation systems to break down problems and accelerate development. Furthermore, empower your "citizen developers" with agent harnesses from day one, coupled with governance to manage duplication. Crucially, build for model and context independence using AI gateways to ensure flexibility and avoid vendor lock-in as technology evolves.

Key insights

Legacy enterprise infrastructure, built for human speeds, is the primary bottleneck for AI agents in production, not the models themselves.

Principles

• Infrastructure designed for humans fails at agent scale.
• LLM-on-LLM evaluation creates shared failure modes.
• Own core infrastructure; use frontier models for specific edge cases.

Method

LinkedIn's approach involves pre-provisioning container pools, scripting 80% of agent workflows with deterministic code, pushing LLMs to the leaf for reasoning, and committing evidence to disk at each step.

In practice

• Prioritize investment in robust evaluation systems.
• Deploy agent harnesses to employees with monitoring.
• Build for model and context independence via AI gateways.

Topics

• AI Agents
• Legacy Infrastructure
• Kubernetes Bottlenecks
• LLM Orchestration
• Data Infrastructure
• Model Governance
• Evaluation Systems

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Engineer, MLOps Engineer, AI Architect

Related on AIssential

• Navigating the AI Infrastructure Landscape: Lessons from Kubernetes Creators — The AI infrastructure landscape requires open platforms and guardrails like MCP to ensure interoperability, safety, and cost control, mirroring cloud-native lessons.
• Why AI’s Deployment Problem May Create the Next Infrastructure Giants — AI's next major bottleneck is deployment and inference, not just model training, driving demand for specialized, distributed infrastructure.
• 'We have maybe 20 months' to rebuild for AI agents, Meta's infrastructure VP tells VB Transform 2026 — Enterprise infrastructure must rapidly adapt to agentic AI, which is fundamentally changing data consumption and system loads.
• Six Sessions at QCon AI Boston 2026 That Take Productionizing AI Seriously — The core challenge in productionizing AI agents lies in robust system engineering beyond model capabilities.
• What it takes to scale agentic AI in the enterprise — Scaling agentic AI demands robust governance, data infrastructure, and system integration, not just advanced technology.
• not much happened today — AI performance increasingly depends on context pipelines and open harnesses, not just model weights.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by VentureBeat.