Towards Speed-of-Light Text Generation with Nemotron-Labs Diffusion Language Models

2026-05-19 · Source: Hugging Face - Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, medium

Summary

NVIDIA has introduced Nemotron-Labs Diffusion, a new family of language models designed to accelerate text generation beyond traditional autoregressive (AR) methods. These Diffusion Language Models (DLMs) generate multiple tokens in parallel and iteratively refine them, addressing AR limitations like token-by-token generation and lack of revision. The collection includes 3B, 8B, and 14B text models under the NVIDIA Nemotron Open Model License, plus an 8B vision-language model. Nemotron-Labs Diffusion supports three generation modes: standard autoregressive, block-by-block diffusion, and self-speculation, which drafts candidates using diffusion and verifies them with AR decoding. The 8B model shows 1.2% improved accuracy over Qwen3 8B. Performance benchmarks indicate diffusion mode achieves 2.6x higher tokens per forward pass (TPF) than AR models, while self-speculation reaches 6x to 6.4x TPF. Training involved a joint AR and diffusion objective on 1.3T pretraining tokens and 45B fine-tuning tokens.

Key takeaway

For AI Engineers building latency-sensitive applications or optimizing GPU utilization, Nemotron-Labs Diffusion models offer a compelling alternative to traditional autoregressive LLMs. You should explore its three generation modes—autoregressive, diffusion, and self-speculation—to achieve up to 6.4x higher tokens per forward pass. This allows you to draft, refine, and verify text generation more efficiently, potentially reducing compute requirements and improving application responsiveness without altering your existing workflows.

Key insights

Diffusion Language Models (DLMs) enable parallel, iterative text generation and revision, significantly boosting speed and accuracy over autoregressive models.

Principles

• Parallel generation improves GPU utilization.
• Iterative refinement allows token revision.
• Joint AR/diffusion training preserves capabilities.

Method

Convert pretrained autoregressive models to diffusion language models through continued pretraining, altering attention to a block-wise approach, and training with a joint AR and diffusion objective.

In practice

• Use diffusion mode for raw throughput.
• Employ self-speculation for lossless AR output at high speed.
• Reduce refinement steps to control inference budget.

Topics

• Diffusion Language Models
• Autoregressive Decoding
• Self-speculation
• LLM Inference Optimization
• NVIDIA Nemotron
• SGLang

Code references

• NVIDIA-NeMo/Megatron-Bridge
• sgl-project/sglang

Best for: NLP Engineer, MLOps Engineer, Research Scientist, AI Engineer, Machine Learning Engineer, AI Scientist

Related on AIssential

• NVIDIA AI Releases Nemotron-Labs-Diffusion: A Tri-Mode Language Model with 6× Tokens Per Forward Over Qwen3-8B — Nemotron-Labs-Diffusion unifies AR and diffusion for efficient, accurate LLM inference via self-speculation.
• Nemotron-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context — Decoupling context and denoising in diffusion models significantly boosts generation throughput while maintaining quality.
• 😸 Diffusion models are coming for text at ~$0.80 per MILLION flat — Diffusion models can significantly accelerate text processing by editing full drafts in parallel, rather than generating word-by-word.
• SAID: Accelerating Diffusion-Based Language Models via Scaffold-Aware Iterative Decoding — SAID accelerates DLLMs by prioritizing denoising on structural tokens and adaptively processing uncertain tokens.
• Run DiffusionGemma on NVIDIA for Developer-Ready, High-Throughput Text Generation — DiffusionGemma employs diffusion-based denoising to generate text tokens in parallel, achieving high throughput for real-time AI.
• Diffusion LLMs from the Ground Up: Theory, Math, and Why They Work — Diffusion LLMs offer a parallel, bidirectional alternative to slow, unidirectional autoregressive text generation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Hugging Face - Blog.