How the NVIDIA Vera Rubin Platform is Solving Agentic AI’s Scale-Up Problem

2026-05-14 · Source: NVIDIA Technical Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Cloud Computing & IT Infrastructure · Depth: Expert, medium

Summary

NVIDIA has introduced the Vera Rubin Platform, featuring the NVIDIA Groq 3 LPX and Vera Rubin NVL72, to address the complex demands of agentic inference workloads. These workloads, characterized by non-deterministic trajectories and multi-turn requests, require sustained low-latency and high-throughput generation for trillion-parameter Mixture-of-Experts (MoE) models with long context windows. Traditional data center fabrics struggle with the small batches and extreme low-latency needs of premium AI services. The Groq 3 LPX, an LPU C2C accelerator, achieves predictable scale-up networking through high-radix point-to-point links, compiler-scheduled data movement, and hardware-driven plesiosynchronous timing. This co-design enables rack-scale determinism, providing 128 GB of unified on-chip SRAM and up to 35x higher throughput per megawatt than NVIDIA GB200 NVL72 for agentic workloads.

Key takeaway

For CTOs or VPs of Engineering evaluating infrastructure for advanced agentic AI services, the NVIDIA Vera Rubin Platform offers a novel solution. Its co-designed Groq 3 LPX and Vera Rubin NVL72 components deliver predictable low-latency and high-throughput for trillion-parameter MoE models, potentially unlocking up to 10x more revenue opportunity. You should consider this platform to overcome the throughput-latency tradeoff in demanding multi-agent deployments.

Key insights

Extreme co-design of hardware and software is crucial for economically scaling agentic AI workloads requiring low-latency and high-throughput.

Principles

• Deterministic execution across chips is vital for agentic AI.
• Networking fabric must be co-designed with silicon and software.
• Static scheduling outperforms runtime arbitration for low-latency.

Method

The LPU C2C extends deterministic execution across many LPUs using high-radix point-to-point links, compiler-scheduled data movement, and hardware-driven plesiosynchronous timing to synchronize thousands of chips.

In practice

• Utilize LPU C2C for multi-trillion parameter MoE models.
• Implement Attention-FFN Disaggregation for heterogeneous compute.
• Orchestrate decode loops with NVIDIA Dynamo for optimized latency.

Topics

• Agentic AI
• NVIDIA Vera Rubin Platform
• NVIDIA Groq 3 LPX
• LPU C2C Interconnect
• Deterministic Execution

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Architect, MLOps Engineer, AI Hardware Engineer

Related on AIssential

• The Rubin Era: How NVIDIA’s New Platform Rewrites the Rules for MoE and Agentic AI — NVIDIA's Vera Rubin platform redefines AI hardware by co-designing six chips for unprecedented performance in MoE and agentic AI.
• Inside NVIDIA Groq 3 LPX: The Low-Latency Inference Accelerator for the NVIDIA Vera Rubin Platform — Heterogeneous AI inference architectures combining GPUs and LPUs significantly boost both throughput and low-latency responsiveness.
• Nvidia: Star Attraction at CES 2026 — Nvidia's Vera Rubin platform integrates co-designed chips and software to build scalable AI factories.
• The NVIDIA Rubin Platform: Six New Chips, One AI Supercomputer — The Vera Rubin architecture integrates co-designed CPUs, GPUs, and DPUs for unprecedented AI supercomputing performance and scalability.
• NVIDIA Vera CPU Sets a New Standard for Agentic Workloads in AI Factories — The NVIDIA Vera CPU optimizes agentic AI and reinforcement learning by making CPU execution a high-performance, critical path within the AI loop.
• GTC Review: NemoClaw, Groq, and SpectrumX — Nvidia's GTC unveiled significant advancements, including the rapid integration of Groq's V3 LPX into a new system platform, utilizing NVLink and a modified SpectrumX for low-latency networking, along…

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Technical Blog.