Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI

2026-07-21 · Source: NVIDIA Technical Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Expert, long

Summary

The NVIDIA Rubin GPU architecture and Vera Rubin platform are engineered to power agentic AI workflows, delivering up to 10x more agentic throughput per unit of energy compared to NVIDIA Blackwell. The Rubin GPU features 336 billion transistors, 224 streaming multiprocessors, 896 Tensor Cores, and a third-generation Transformer Engine providing up to 50 petaflops of NVFP4 inference performance. It integrates up to 288 GB of HBM4 memory with 22 TB/s peak bandwidth, alongside NVLink 6 and PCIe Gen 6 for high-speed communication. Key architectural innovations include enhanced Tensor Memory Accelerator for Mixture-of-Experts models, doubled K-dimension Tensor Core throughput, and accelerated long-context attention via activation sparsity and improved softmax. The Vera Rubin NVL72 rack-scale system further optimizes efficiency with liquid cooling, Intelligent Power Smoothing, and DSX MaxLPS, enabling up to 40% more GPUs within the same power budget.

Key takeaway

For AI Architects designing next-generation agentic AI infrastructure, the NVIDIA Rubin platform offers significant advancements in throughput and power efficiency. You should evaluate Rubin's 10x performance uplift over Blackwell, especially its HBM4 memory, enhanced Transformer Engine, and rack-scale power smoothing, to optimize for sustained, low-latency inference and maximize GPU density within your power budget. Consider integrating DSX MaxLPS to provision up to 40% more GPUs.

Key insights

Rubin GPU architecture focuses on end-to-end efficiency for agentic AI, optimizing compute, memory, and rack-scale power.

Principles

• Agentic AI demands sustained, low-latency inference.
• Data movement efficiency is critical for large models.
• Rack-scale power management boosts GPU density.

Method

Rubin GPU uses reticle-limited compute dies unified by NV-HBI, organizing resources into GPCs with a large L2 cache and third-gen Transformer Engine for precision adaptation.

In practice

• Utilize HBM4 capacity for larger KV caches and context windows.
• Apply activation sparsity in attention blocks for efficiency.
• Implement power smoothing techniques for higher GPU density.

Topics

• NVIDIA Rubin GPU
• Agentic AI
• AI Inference Optimization
• HBM4 Memory
• Transformer Engine
• Data Center Power Efficiency

Best for: MLOps Engineer, CTO, VP of Engineering/Data, AI Hardware Engineer, AI Architect, Machine Learning Engineer

Related on AIssential

• Inside the NVIDIA Rubin Platform: Six New Chips, One AI Supercomputer — NVIDIA's Rubin platform uses extreme co-design to optimize entire data centers for continuous, scalable AI intelligence production.
• Now Generally Available, NVIDIA RTX PRO 5000 72GB Blackwell GPU Expands Memory Options for Desktop Agentic AI — The NVIDIA RTX PRO 5000 72GB Blackwell GPU expands desktop AI capabilities with increased memory for larger models.
• Inside NVIDIA Groq 3 LPX: The Low-Latency Inference Accelerator for the NVIDIA Vera Rubin Platform — Heterogeneous AI inference architectures combining GPUs and LPUs significantly boost both throughput and low-latency responsiveness.
• The Agentic CPU Turn — Agentic AI shifts compute demand from GPU-heavy inference to CPU-intensive orchestration, fundamentally altering AI infrastructure design.
• The Rubin Era: How NVIDIA’s New Platform Rewrites the Rules for MoE and Agentic AI — NVIDIA's Vera Rubin platform redefines AI hardware by co-designing six chips for unprecedented performance in MoE and agentic AI.
• Jensen Huang just put Nvidia’s Blackwell and Vera Rubin sales projections into the $1 trillion stratosphere — Nvidia projects $1 trillion in Blackwell and Rubin chip orders by 2027, reflecting booming AI hardware demand.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Technical Blog.