DiffusionGemma 26B A4B

· vllm · 256K context · DGX Spark GB10 · 128 GB

Google's discrete-diffusion Gemma 4 26B A4B: decodes blocks of tokens in parallel, with vision and thinking.

56 tok/sdecode

463 tok/sprefill

256Kcontext window

Reported by toncaoSep 30, 2026

Run it

This is the launch as toncao publishes it. Their repository has the full setup: toncao/local-ai-registry@f5be8bf9c62b449a7fe1c82ce01f9bd10426e861 › The server then answers on http://localhost:8000/v1.

1 Download the weights

hf download google/diffusiongemma-26B-A4B-it \
  --revision f7f5b7f5fa82ffc52addd066915886d497f5517b \
  --local-dir ~/models/diffusiongemma-26B-A4B-it-f7f5b7f5

2 Start the server

docker run --rm \
  --gpus all \
  -p 8000:8000 \
  --shm-size 16g \
  -v ~/models/diffusiongemma-26B-A4B-it-f7f5b7f5:/models/diffusiongemma-26B-A4B-it-f7f5b7f5:ro \
  --entrypoint vllm \
  vllm/vllm-openai:v0.30.0@sha256:8a69ffad015f138d7170c4ddc429e230a3bc1c1719f67e14324749df200a4b90 \
  serve \
  /models/diffusiongemma-26B-A4B-it-f7f5b7f5 \
  --served-model-name diffusiongemma-26B-A4B-it-f7f5b7f5 \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 262144 \
  --max-num-seqs 4 \
  --gpu-memory-utilization 0.8 \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser gemma4 \
  --reasoning-parser gemma4 \
  --max-num-batched-tokens 8192 \
  --enable-prefix-caching \
  --override-generation-config '{"max_new_tokens": null}' \
  --default-chat-template-kwargs '{"enable_thinking":true}' \
  --attention-backend TRITON_ATTN \
  --safetensors-load-strategy eager

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by toncao in toncao/local-ai-registry@f5be8bf9c62b449a7fe1c82ce01f9bd10426e861, where it reports 55.8 tok/s. Our six checks have not run on it yet. Whole-answer decode; temperature omitted. Canonical lab pending. Requires temperature 1.0; tools 16/20 (final reply sometimes in reasoning_content).

Details

Weights: google/diffusiongemma-26B-A4B-it @ f7f5b7f5fa ›
Image: vllm/vllm-openai:v0.30.0@sha256:8a69ffad015f138d7170c4ddc429e230a3bc1c1719f67e14324749df200a4b90
Source: toncao/local-ai-registry @ f5be8bf9c62b449a7fe1c82ce01f9bd10 ›
Engine profile: vllm ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/diffusiongemma-26b-a4b.vllm.256k.json ›
Model released: Jun 9, 2026
Tested: not yet by us; reported by toncao