Qwen3.6-35B-A3B

· vllm · 128K context · 2 × RTX 3090 in one machine · 48 GB

Qwen3.6's 35B mixture-of-experts model with 3B active parameters.

170 tok/sdecode

– tok/sprefill

128Kcontext window

Earlier checkearlier

Run it

Follow the steps below to download and run this recipe. Omarchy Local AI lists the recipes it can launch on your hardware. The server then answers on http://localhost:8000/v1.

1 Download the weights

hf download Twu31/Qwen3.6-35B-A3B-GPTQ-INT4-W4A16-LowLatency \
  --revision d14fd7339b09364ea2aabc542ea20f7dccd97ca1

2 Start the server

docker run --rm \
  --gpus all \
  -p 8000:8000 \
  --shm-size 32g \
  -e VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai@sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967 \
  --model Twu31/Qwen3.6-35B-A3B-GPTQ-INT4-W4A16-LowLatency \
  --revision d14fd7339b09364ea2aabc542ea20f7dccd97ca1 \
  --served-model-name Qwen3.6-35B \
  --tensor-parallel-size 2 \
  --max-model-len 131072 \
  --max-num-seqs 8 \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.97 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3

What it passed

✓ loadLoads and serves within an hour

✓ chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Passed the older check (loads and chats); a full six-check run is pending.

Details

Weights: Twu31/Qwen3.6-35B-A3B-GPTQ-INT4-W4A16-LowLatency @ d14fd7339b ›
Image: vllm/vllm-openai@sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967
Launch: vllm-qwen3.6-35b-a3b-int4-w4a16-128k-v2 ›
Recipe file: registry/recipes/nvidia/rtx-3090-24gb/qwen3.6-35b-a3b.vllm.128k.2x.json ›
Model released: Apr 15, 2026
Tested: earlier acceptance