GLM-5.3-Flash

NVFP4 · sglang · 1024K context · 4 × DGX Spark GB10 · 512 GB

Z.ai's GLM-5.3-Flash, a large mixture-of-experts model for multi-GPU machines.

59 tok/sdecode

– tok/sprefill

1024Kcontext window

Earlier checkearlier

Run it

Follow the steps below to download and run this recipe. Omarchy Local AI lists the recipes it can launch on your hardware. The server then answers on http://localhost:8000/v1.

1 Download the weights

hf download LibertAIDAI/GLM-5.3-Flash-NVFP4 \
  --revision 9e0d74e3cef17f634e84fb8e2223707e02616290 \
  --local-dir ~/models/GLM-5.3-Flash-NVFP4-9e0d74e3

2 Start the server on each of the 4 machines (NODE_RANK 0 to 3)

docker run --rm \
  --gpus all \
  -p 8000:8000 \
  --ipc host \
  --network host \
  --shm-size 64g \
  -e CUTE_DSL_ARCH=sm_121a \
  -e FLASHINFER_CUDA_ARCH_LIST=12.1a \
  -e NCCL_IB_HCA=rocep1s0f1,roceP2p1s0f1 \
  -e NCCL_NET=IB \
  -e NCCL_SOCKET_IFNAME="${FABRIC_INTERFACE}" \
  -e SGLANG_FP8_PAGED_MQA_LOGITS_TORCH=1 \
  -e SGLANG_GLM53_ONLINE_FP8_LINEAR=1 \
  -e SGLANG_GLM53_ONLINE_FP8_LM_HEAD=0 \
  -e SGLANG_OPT_USE_TILELANG_INDEXER=1 \
  -e TORCH_CUDA_ARCH_LIST=12.1a \
  -v ~/models/GLM-5.3-Flash-NVFP4-9e0d74e3:/model:ro \
  ghcr.io/0xsero/glm-5.3-flash-sglang-sm121@sha256:f9ac60ba4071f8acd64f0f3c074aca308f6d659405fee46fc8031489a1e8b19b \
  --model-path /model \
  --served-model-name glm-5.3-flash \
  --tp-size 4 \
  --ep-size 4 \
  --nnodes 4 \
  --node-rank "${NODE_RANK}" \
  --dist-init-addr '${HEAD_FABRIC_ADDRESS}:27000' \
  --context-length 1048576 \
  --quantization modelopt_fp4 \
  --attention-backend dsa \
  --dsa-prefill-backend flashinfer_sparse_mla \
  --dsa-decode-backend flashinfer_sparse_mla \
  --linear-attn-backend triton \
  --kv-cache-dtype fp8_e4m3 \
  --moe-runner-backend flashinfer_cutlass \
  --disable-shared-experts-fusion \
  --chunked-prefill-size 8192 \
  --max-prefill-tokens 8192 \
  --max-running-requests 8 \
  --mem-fraction-static 0.90 \
  --cuda-graph-max-bs-decode 8 \
  --speculative-algorithm NEXTN \
  --speculative-num-steps 5 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 6 \
  --speculative-adaptive \
  --media-url-max-file-size-mb 1024 \
  --enable-multimodal \
  --chat-template /opt/glm53/chat-template-mm.jinja \
  --reasoning-parser glm45 \
  --tool-call-parser glm47 \
  --host 0.0.0.0 \
  --port 8000

What it passed

✓ loadLoads and serves within an hour

✓ chatAnswers a plain question and stops on its own

✓ reasoningThinks separately and gets 17 × 23 right

✓ toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Passed the older check (loads and chats); a full six-check run is pending.

Details

Weights: LibertAIDAI/GLM-5.3-Flash-NVFP4 @ 9e0d74e3ce ›
Image: ghcr.io/0xsero/glm-5.3-flash-sglang-sm121@sha256:f9ac60ba4071f8acd64f0f3c074aca308f6d659405fee46fc8031489a1e8b19b
Launch: sglang-glm-5.3-flash-nvfp4-1024k-4x ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/glm-5.3-flash.sglang.1024k.4x.json ›
Model released: Aug 25, 2026
Tested: earlier acceptance