Qwen3.6-35B-A3B

NVFP4 · vllm · 256K context · DGX Spark GB10 · 128 GB

Qwen3.6's 35B mixture-of-experts model with 3B active parameters.

95 tok/sdecode

– tok/sprefill

256Kcontext window

Reported by MiaAI-LabJul 29, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/Unsloth-Qwen3.6-35b-NVFP4-DGX-Spark@79c9e6f359f6 › The server then answers on http://localhost:8000/v1.

1 Set up first

# Mounts ./.cache/huggingface -> /root/.cache/huggingface and repo dir -> /workspace. Downloads model on host via hf / huggingface-cli (or python in the image) if not cached. Image must be pulled from GHCR or built on an arm64 Spark with docker/build.sh.
# Full instructions: https://github.com/MiaAI-Lab/Unsloth-Qwen3.6-35b-NVFP4-DGX-Spark/tree/79c9e6f359f6101cdacd0dfd6fe9861ae2493a4d

2 Download the weights

hf download unsloth/Qwen3.6-35B-A3B-NVFP4 \
  --revision 739af1e7aac320af1682ed1e0cce369af4c5265d

3 Start the server

docker run --rm \
  --gpus all \
  -p 8000:8888 \
  --user root \
  --network host \
  --shm-size=32g \
  --ulimit memlock=-1:-1 \
  --cap-add=IPC_LOCK \
  --ipc host \
  --gpus all \
  --workdir /workspace \
  --shm-size 32g \
  -e CUTE_DSL_ARCH=sm_121a \
  -e HF_HOME=/root/.cache/huggingface \
  -e HF_TOKEN='${HF_TOKEN:-}' \
  -e VLLM_TARGET_DEVICE=cuda \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint /usr/local/bin/vllm \
  ghcr.io/miaai-lab/mia-vllm-gb10-linear-b12x:latest@sha256:19627342e1da2607f4db50745dca30e57d7dd0ebff06062f03fd69b43a252931 \
  serve \
  unsloth/Qwen3.6-35B-A3B-NVFP4 \
  --host 0.0.0.0 \
  --port 8888 \
  --tensor-parallel-size 1 \
  --trust-remote-code \
  --moe-backend auto \
  --gpu-memory-utilization 0.80 \
  --linear-backend flashinfer_b12x \
  --attention-backend flashinfer \
  --max-model-len 262144 \
  --max-num-seqs 24 \
  --max-num-batched-tokens 32768 \
  --enable-chunked-prefill \
  --async-scheduling \
  --kv-cache-dtype fp8 \
  --limit-mm-per-prompt '{"image":4}' \
  --allowed-media-domains '*' \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2,"moe_backend":"triton"}' \
  --reasoning-parser qwen3 \
  --default-chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
  --tool-call-parser qwen3_coder \
  --enable-auto-tool-choice \
  --override-generation-config '{"temperature":0.6,"top_p":0.95,"top_k":20,"min_p":0.0,"presence_penalty":0.0,"repetition_penalty":1.0}'

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/Unsloth-Qwen3.6-35b-NVFP4-DGX-Spark@79c9e6f359f6, where it reports 95.1 tok/s. Our six checks have not run on it yet.

Details

Weights: unsloth/Qwen3.6-35B-A3B-NVFP4 @ 739af1e7aa ›
Image: ghcr.io/miaai-lab/mia-vllm-gb10-linear-b12x:latest@sha256:19627342e1da2607f4db50745dca30e57d7dd0ebff06062f03fd69b43a252931
Source: MiaAI-Lab/Unsloth-Qwen3.6-35b-NVFP4-DGX-Spark @ 79c9e6f359f6 ›
Launch: vllm-qwen3.6-35b-a3b-nvfp4-256k ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/qwen3.6-35b-a3b.vllm.256k.json ›
Model released: Apr 15, 2026
Tested: not yet by us; reported by miaai-lab