Google’s latest DiffusionGemma open AI model comes with a 4x speed boost

In a notable advancement in artificial intelligence, Google DeepMind has introduced DiffusionGemma, a groundbreaking addition to the Gemma 4 open model family. This innovative model departs from traditional linear text generation methods, enabling it to produce entire blocks of text simultaneously. The key to this enhanced speed and efficiency lies in DiffusionGemma's unique architecture, which allows it to operate effectively on local hardware, including Nvidia DGX systems and standard gaming GPUs. Unlike most AI models that generate text in a sequential, left-to-right manner, DiffusionGemma employs a technique reminiscent of image generation models. It begins with a field of placeholder tokens and iteratively refines them to produce the final output, resulting in what is termed a “denoised” text canvas. With 26 billion parameters, DiffusionGemma is classified as a Mixture of Experts (MoE) model. However, only 3.8 billion parameters are activated during inference, ensuring it can run efficiently within the 18GB RAM limits of high-end GPUs. In testing scenarios using an RTX 5090, the model achieves an impressive output of approximately 700 tokens per second. When paired with a single Nvidia H100 AI accelerator, it can exceed 1,000 tokens per second—about four times the output of its autoregressive counterparts. This innovative approach shifts the focus from memory bandwidth limitations to computational power, allowing for the generation of up to 256 tokens in parallel. Google highlights that this capability significantly enhances performance in complex tasks such as in-line editing, molecular sequencing, and mathematical graphing. A demonstration of DiffusionGemma's prowess is evident in its ability to solve Sudoku puzzles—an area where standard autoregressive models often struggle due to the interdependencies of tokens. By continuously self-correcting large sets of tokens, DiffusionGemma streamlines this challenging process.