MLPerf Training v6.0: Lambda delivers fastest LLM training on NVIDIA GB300 NVL72 and fastest MoE training on NVIDIA HGX B200

2026-06-16 · Source: The Lambda Deep Learning Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Advanced, short

Summary

Lambda achieved leading performance in the MLPerf Training v6.0 benchmarks, specifically for large language model (LLM) and mixture-of-experts (MoE) training. On the NVIDIA GB300 NVL72 system, Lambda's Llama 3.1 8B training completed in 11.59 minutes, an 18.7% improvement over its previous v5.1 result and the fastest among GB300 NVL72 submissions. For MoE models, Lambda recorded the fastest single-node HGX B200 result for GPT-OSS-20B at 96.46 minutes, and a competitive 18.35 minutes on GB300 NVL72. MLPerf Training v6.0 notably expanded its suite to include MoE models like OpenAI's GPT-OSS-20B, reflecting real-world AI training workloads. These gains are attributed to NVIDIA Blackwell Ultra innovations, including 279GB HBM3e per GPU, enhanced memory bandwidth, and improved interconnects, combined with Lambda's cluster design and optimized NVIDIA NeMo software stack.

Key takeaway

For AI Architects evaluating infrastructure for large-scale LLM or MoE training, Lambda's MLPerf v6.0 results demonstrate validated performance. Your teams can achieve significant time savings, potentially days or weeks, by leveraging NVIDIA GB300 NVL72 systems with optimized software like NVIDIA NeMo. Consider Lambda's 1-Click Clusters for benchmarking new architectures or scaling production runs, ensuring your hardware decisions are based on reproducible, industry-standard conditions.

Key insights

Optimized hardware and software stacks significantly accelerate LLM and MoE model training convergence.

Principles

• Training performance depends on architecture, system design, and software.
• MLPerf benchmarks provide reproducible training speed baselines.
• MoE architectures introduce distinct training challenges.

In practice

• Evaluate NVIDIA GB300 NVL72 for frontier-scale MoE training.
• Utilize NVIDIA NeMo software stack for performance gains.
• Benchmark new architectures against MLPerf standards.

Topics

• MLPerf Training
• LLM Training
• Mixture-of-Experts
• NVIDIA GB300 NVL72
• NVIDIA HGX B200
• NVIDIA NeMo

Best for: MLOps Engineer, AI Engineer, NLP Engineer, Machine Learning Engineer, AI Architect, Director of AI/ML

Related on AIssential

• DeepSeek-V3: A Large-Scale MoE Pretraining Benchmark for MLPerf Training v6.0 — MLPerf Training v6.0 now benchmarks MoE LLMs, reflecting industry shifts to sparse computation and specialized architectures.
• Lambda’s NVIDIA HGX 8xB200 on STAC-AI™ LANG6 — NVIDIA HGX 8xB200 significantly outperforms H200 NVL in LLM inference latency and throughput, especially under load and for large models.
• Reproducing AMD MLPerf Training v6.0 Submission Result — AMD provides a detailed guide to reproduce its MLPerf Training v6.0 benchmark results on Instinct GPUs using the Primus framework.
• NVIDIA Extreme Co-Design Delivers New MLPerf Inference Records — Co-designed hardware, software, and models are crucial for maximizing AI factory throughput and minimizing token cost.
• Boosting MoE Training Throughput with Advanced Fusion Kernels — Fusing MoE block operations with custom kernels significantly boosts training throughput by eliminating bottlenecks.
• 2026 Predictions: Much Faster Inference, Pre-Training with RL, and FP4 Everywhere — DeepSeek's MoE and RLVR advancements, alongside FP4 quantization, defined 2025 LLM progress, while Llama 4 disappointed.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Lambda Deep Learning Blog.