“DiffusionGemma: Google AI’s 26B MoE Open Model Achieves 4x Faster Generation with Text Diffusion”

Google’s AI division, along with researchers from Google DeepMind, has unveiled DiffusionGemma — an experimental open-source model designed for text generation. Unlike conventional approaches, it relies on text diffusion rather than traditional autoregressive decoding. Released under the flexible Apache 2.0 license, Google is targeting developers and researchers who need fast, interactive local workflows. Think real-time in-line editing, quick prototyping, and building non-linear text outputs.

The vast majority of today’s language models follow an autoregressive approach. They produce tokens sequentially, moving from left to right, with each new token conditioned on the one before it. DiffusionGemma takes a different path. It generates entire chunks of text all at once, in parallel. On capable GPUs, this translates to generation speeds up to four times faster.

What is DiffusionGemma

DiffusionGemma is a 26-billion-parameter Mixture of Experts (MoE) model. At inference time, only 3.8 billion parameters are active. It’s built on the Gemma 4 foundation — specifically the 26B-A4B architecture — with a diffusion head layered on top.

The model handles multiple modalities. It accepts mixed text, image, and video inputs and produces text outputs from them. It supports a context window of 256,000 tokens and works across more than 140 languages.

In its quantized form, the model fits within 18 GB of VRAM — well within the range of high-end consumer GPUs. On a single NVIDIA H100, it can push past 1,000 tokens per second. On an NVIDIA GeForce RTX 5090, it hits over 700 tokens per second.

Google is upfront about the compromise. DiffusionGemma is optimized for speed and parallel layout generation. Its overall output quality falls short of the standard Gemma 4. For production tasks where quality is paramount, Google still points users toward the autoregressive Gemma 4.

How Text Diffusion Works

Text diffusion takes its core concept from AI image generators. Those systems begin with visual noise and progressively refine it into a coherent image. DiffusionGemma applies that same iterative refinement process to text.

The process unfolds in three broad stages. First, the model begins with a canvas filled with random placeholder tokens. Second, it runs multiple refinement passes over that canvas, locking in high-confidence tokens and using them as anchors. Third, the text gradually converges into the finished output.

Google refers to the core mechanism as Uniform State Diffusion. Tokens with high confidence help resolve neighboring positions during the denoising process. Over successive passes, the full sequence sharpens into focus.

In practice, the model denoises a 256-token canvas in parallel, finalizing approximately 15 to 20 tokens with each forward pass. That parallelism is the engine behind the throughput improvements.

During denoising, the model employs bidirectional attention. Every token on the canvas can reference every other token. This is a fundamental departure from autoregressive models, which can only look backward at previously generated tokens.

That bidirectional context allows for real-time self-correction. If a token’s confidence drops, the sampler can re-noise it and replace it on a later pass. Autoregressive models lack this ability — once a token is generated, it’s locked in.

The Architecture

The key technical innovation here is how it leverages hardware. For local GPU inference, the primary bottleneck is memory bandwidth. Autoregressive models repeatedly fetch weights from memory for each token. During single-user serving, the GPU spends most of its time idle, waiting on data.

DiffusionGemma shifts that bottleneck from memory bandwidth to compute. By drafting and refining a 256-token canvas in parallel, it gives idle tensor cores a substantial parallel workload to chew on.

The model switches between two attention modes during inference. The prefill phase uses causal attention to process the prompt and populate the KV cache. The denoising phase uses bidirectional attention to refine the canvas.

For longer outputs, DiffusionGemma employs Block Autoregressive Diffusion. Once a 256-token block is fully denoised, it’s committed to the KV cache. The model then begins a fresh canvas conditioned on all prior context. This combines the speed of parallel block generation with the stability of sequential autoregressive coherence.

The architecture shares the same backbone as Gemma 4 26B A4B. For developers, the main addition is implementing a denoising step, which makes integration into existing serving frameworks relatively straightforward.

A telling example comes from Google’s developer guide: a Sudoku puzzle showcase. Autoregressive models struggle with rigid, multi-variable constraint puzzles. The base DiffusionGemma model solves roughly 0% of Sudoku puzzles. After applying a straightforward JAX supervised fine-tuning recipe, accuracy jumps to 80%. The fine-tuned model also converges faster, reducing the number of inference steps needed.

Interactive Demo: How DiffusionGemma Decodes in Parallel

The interactive visualizer below demonstrates how DiffusionGemma decodes text, compared side by side with a standard autoregressive model. Switch between the two modes and hit Run. In Autoregressive mode, tokens appear one at a time, strictly from left to right, requiring one forward pass per token — the way most LLMs operate today. In Diffusion mode, the model starts from a canvas of masked placeholder tokens and resolves many of them simultaneously each pass, in no particular order, converging in significantly fewer passes. The animation also includes a brief re-noise step, where a low-confidence token is reset and refined again — representing the real model’s self-correction capability, something autoregressive decoding can’t do once a token is committed. Keep in mind this is a conceptual animation, not live model output: the actual DiffusionGemma processes a 256-token canvas and finalizes roughly 15–20 tokens per forward pass.

Use Cases

DiffusionGemma is aimed at specific workloads rather than general-purpose production quality. Google and its ecosystem partners highlight several practical applications:

• In-line editing and code infilling: Bidirectional attention is well-suited to non-linear text structures.
• Rapid iteration: Low local latency supports interactive, single-user developer workflows.
• Long-context document analysis: The 256K window accommodates processing large inputs.
• OCR and document parsing: The model’s parallel generation strengths align well with structured extraction tasks.