Qwen3.8-Flash-Next

NVFP4 · vllm · 256K context · 2 × DGX Spark GB10 · 256 GB

Qwen3.8's large mixture-of-experts model: far more knowledge than the 27B, with few active parameters, so it stays fast.

54 tok/sdecode

2,962 tok/sprefill

256Kcontext window

Reported by MiaAI-LabSep 16, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/Qwen3.8-Flash-Next-Dual-DGX-Sparks@d2f54b78c0d2 › The server then answers on http://localhost:8000/v1.

1 Set up first

# Passwordless SSH head->worker, ConnectX RoCE link (IFACE/IB_HCA in .env). start.sh rsyncs (or NFS-shares) weights to worker, extracts and patches vLLM files from the image (ple_layer.py, modelopt.py for MXFP8 + FP8_BLOCK_SCALES MTP experts, mtp.py draft vocab), patches checkpoint config (mtp.layers.48 alias) and bind-mounts them ro; evicts checkpoint page cache before launch. Worker (rank 1, --headless) starts first, head (rank 0) serves.
# Full instructions: https://github.com/MiaAI-Lab/Qwen3.8-Flash-Next-Dual-DGX-Sparks/tree/d2f54b78c0d2f9d74ac61aa56200e3c40fac3f22

2 Download the weights

hf download nvidia/Qwen3.8-Flash-Next-NVFP4 \
  --revision fc694b54fb0174e0913e6adf86691ef85a4ead47

3 Start the server on each of the 2 machines (NODE_RANK 0 to 1)

docker run --rm \
  --gpus all \
  -p 8000:8888 \
  --gpus all \
  --network host \
  --ipc host \
  --cap-add SYS_NICE \
  --ulimit memlock=-1 \
  --ulimit stack=67108864 \
  --device /dev/infiniband:/dev/infiniband \
  -e GLOO_SOCKET_IFNAME="${IFACE}" \
  -e HF_HOME=/root/.cache/huggingface \
  -e HF_HUB_OFFLINE=1 \
  -e NCCL_DEBUG=WARN \
  -e NCCL_IB_AUTO_DETECT=0 \
  -e NCCL_IB_DISABLE=0 \
  -e NCCL_IB_GID_INDEX=3 \
  -e NCCL_IB_HCA="${IB_HCA}" \
  -e NCCL_SOCKET_IFNAME="${IFACE}" \
  -e TP_SOCKET_IFNAME="${IFACE}" \
  -e TRANSFORMERS_OFFLINE=1 \
  -e VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
  -e VLLM_HOST_IP="${NODE_IP}" \
  -e VLLM_MTP_DRAFT_VOCAB=/etc/vllm-draft-vocab.txt \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:qwen38-flash-next@sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8 \
  nvidia/Qwen3.8-Flash-Next-NVFP4 \
  --served-model-name qwen3.8-flash-next \
  --tensor-parallel-size 2 \
  --gpu-memory-utilization 0.835 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 8192 \
  --max-model-len 262144 \
  --kv-cache-dtype fp8 \
  --mamba-ssm-cache-dtype bfloat16 \
  --load-format safetensors \
  --safetensors-load-strategy lazy \
  --enable-chunked-prefill \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --distributed-executor-backend mp \
  --mm-encoder-tp-mode data \
  --nnodes 2 \
  --master-addr "${HEAD_IP}" \
  --master-port 50000 \
  --enable-expert-parallel \
  --all2all-backend allgather_reducescatter \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3,"use_local_argmax_reduction":true}' \
  --compilation-config '{"mode":0,"cudagraph_mode":"FULL_DECODE_ONLY"}' \
  --hf-overrides '{"text_config":{"ple_embedding_dtype":"${PLE_EMBEDDING_DTYPE}"}}' \
  --node-rank "${NODE_RANK}"

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/Qwen3.8-Flash-Next-Dual-DGX-Sparks@d2f54b78c0d2, where it reports 54.4 tok/s. Our six checks have not run on it yet.

Details

Weights: nvidia/Qwen3.8-Flash-Next-NVFP4 @ fc694b54fb ›
Image: vllm/vllm-openai:qwen38-flash-next@sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8
Source: MiaAI-Lab/Qwen3.8-Flash-Next-Dual-DGX-Sparks @ d2f54b78c0d2f ›
Launch: vllm-qwen3.8-flash-next-nvfp4-256k-2x ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/qwen3.8-flash-next.vllm.256k.2x.json ›
Model released: Aug 24, 2026
Tested: not yet by us; reported by miaai-lab