Qwen3.6-27B

NVFP4 · vllm · 256K context · DGX Spark GB10 · 128 GB

Alibaba's dense Qwen3.6, 27B parameters.

46 tok/sdecode

1,083 tok/sprefill

256Kcontext window

Reported by MiaAI-LabJul 31, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/Qwen3.6-27B-NVFP4-DFlash-DGX-Spark@c9b071012e8e › The server then answers on http://localhost:8000/v1.

1 Set up first

# Mounts ./.cache/huggingface -> /root/.cache/huggingface, ./.cache/triton -> /root/.triton, ./chat_template.jinja -> /workspace/chat_template.jinja and the repo dir -> /workspace. Custom chat template qwen3.6-froggeric-v20 (froggeric/Qwen-Fixed-Chat-Templates) is required. Optional HF_TOKEN.
# Full instructions: https://github.com/MiaAI-Lab/Qwen3.6-27B-NVFP4-DFlash-DGX-Spark/tree/c9b071012e8e7a74cdb5277413332d0767ba7fff

2 Download the weights

hf download nvidia/Qwen3.6-27B-NVFP4 \
  --revision 0893e1606ff3d5f97a441f405d5fc541a6bdf404

3 Start the server

docker run --rm \
  --gpus all \
  -p 8000:8888 \
  --network host \
  --ipc host \
  --privileged \
  --gpus all \
  -e CUTE_DSL_ARCH=sm_121a \
  -e HF_HOME=/root/.cache/huggingface \
  -e HF_TOKEN='${HF_TOKEN:-}' \
  -e TRITON_CACHE_DIR=/root/.triton \
  -e VLLM_FLOAT32_MATMUL_PRECISION=high \
  -e VLLM_TARGET_DEVICE=cuda \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint vllm \
  vllm/vllm-openai:nightly-aarch64@sha256:2352b4a6a8f290967eed33fad5946c94c668479f6fa5816c13d26ef7aa13f889 \
  serve \
  nvidia/Qwen3.6-27B-NVFP4 \
  --served-model-name qwen36-27b-nvidia-nvfp4-dflash \
  --host 0.0.0.0 \
  --port 8888 \
  --tensor-parallel-size 1 \
  --trust-remote-code \
  --quantization compressed-tensors \
  --moe-backend flashinfer_b12x \
  --attention-backend flash_attn \
  --kv-cache-dtype bfloat16 \
  --gpu-memory-utilization 0.84 \
  --max-model-len 262144 \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --skip-mm-profiling \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --enable-auto-tool-choice \
  --generation-config vllm \
  --speculative-config '{"method":"dflash","model":"z-lab/Qwen3.6-27B-DFlash","num_speculative_tokens":10}' \
  --chat-template /workspace/chat_template.jinja \
  --default-chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}'

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/Qwen3.6-27B-NVFP4-DFlash-DGX-Spark@c9b071012e8e, where it reports 45.6 tok/s. Our six checks have not run on it yet.

Details

Weights: nvidia/Qwen3.6-27B-NVFP4 @ 0893e1606f ›
Image: vllm/vllm-openai:nightly-aarch64@sha256:2352b4a6a8f290967eed33fad5946c94c668479f6fa5816c13d26ef7aa13f889
Source: MiaAI-Lab/Qwen3.6-27B-NVFP4-DFlash-DGX-Spark @ c9b071012e8e7 ›
Launch: vllm-qwen3.6-27b-nvfp4-256k ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/qwen3.6-27b.vllm.256k.json ›
Model released: Apr 21, 2026
Tested: not yet by us; reported by miaai-lab