NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI

2026-06-10 · Source: NVIDIA Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Cloud Computing & IT Infrastructure · Depth: Intermediate, short

Summary

Google DeepMind has released DiffusionGemma, an experimental open model designed for exceptionally fast text generation, which NVIDIA has optimized for its GeForce RTX GPUs, RTX PRO platform, and DGX Spark systems. Unlike traditional autoregressive large language models that generate text one token at a time, DiffusionGemma employs a diffusion-based approach, generating up to 256 tokens in parallel per step. Built on the Gemma 4 26-billion-parameter mixture-of-experts architecture, activating 3.8 billion parameters per step, this model achieves up to 4x faster performance for single-user, latency-sensitive workloads on local hardware. It operates under a permissive Apache 2.0 license, supports day-zero integration with Hugging Face Transformers, vLLM, and Unsloth, and delivers impressive speeds like 1,000 tokens/sec on an NVIDIA H100 GPU.

Key takeaway

For AI Engineers and researchers developing latency-sensitive, single-user text generation applications, DiffusionGemma offers a significant performance advantage. You should consider integrating this open-weights model to achieve up to 4x faster local inference on NVIDIA GPUs, reducing reliance on cloud resources and per-token costs. Explore its day-zero support in Hugging Face Transformers or vLLM for rapid prototyping and deployment.

Key insights

DiffusionGemma uses parallel diffusion for text generation, achieving 4x faster local inference than autoregressive LLMs.

Principles

• Parallel text generation outperforms sequential for speed.
• Diffusion models can adapt beyond image generation.
• Compute-bound workloads benefit most from GPUs.

Method

DiffusionGemma generates text by denoising blocks of up to 256 tokens in parallel, starting from noise, leveraging a diffusion head with Gemma 4 architecture.

In practice

• Run DiffusionGemma locally on RTX or DGX Spark.
• Use Hugging Face Transformers for quick prototyping.
• Fine-tune with Unsloth or NVIDIA NeMo framework.

Topics

• DiffusionGemma
• Parallel Text Generation
• NVIDIA GPU Optimization
• Local AI Inference
• Gemma 4 Architecture
• Open-source Models

Best for: NLP Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• Google's new open model DiffusionGemma generates text from noise instead of word by word — DiffusionGemma generates text in parallel blocks from noise, significantly boosting local inference speed for non-linear tasks.
• Run DiffusionGemma on NVIDIA for Developer-Ready, High-Throughput Text Generation — DiffusionGemma employs diffusion-based denoising to generate text tokens in parallel, achieving high throughput for real-time AI.
• Google's DiffusionGemma generates 256 tokens in parallel and self-corrects as it goes — DiffusionGemma applies image diffusion to text, enabling parallel, self-correcting block generation for faster inference in specific contexts.
• DiffusionGemma — Google's experimental Gemini Diffusion model is now an open-weight, Apache 2 licensed DiffusionGemma.
• Systematic Optimization of Real-Time Diffusion Model Inference on Apple M3 Ultra — Apple Silicon's unified memory architecture requires distinct diffusion model optimization strategies compared to NVIDIA GPUs.
• A Visual Guide to DiffusionGemma — DiffusionGemma uses iterative diffusion to generate text, optimizing single-user latency by processing 256 tokens concurrently.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Blog.