Qwen3.8-27B

EXL3 3.5 bpw · exllamav3 · 256K context · DGX Spark GB10 · 128 GB

Alibaba's 27B dense model from August 2026, and the strongest model that fits one consumer card. It thinks before it answers, calls tools reliably and reads images, which makes it the default for coding agents and long projects.

48 tok/sdecode

– tok/sprefill

256Kcontext window

Reported by MiaAI-LabSep 1, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw@1242187390f7 › The server then answers on http://localhost:8888/v1.

1 Set up first

# Not Docker. start.sh self-bootstraps: python3 -m venv .venv; pip install torch --extra-index-url ${TORCH_INDEX_URL:-https://download.pytorch.org/whl/cu130}; pip install --no-build-isolation ${EXL3_REPO:-git+https://github.com/MiaAI-Lab/exllamav3} (CUDA kernel compile, 5-20 min; TORCH_CUDA_ARCH_LIST=12.0;12.1 auto-set on aarch64; MAX_JOBS 8 or 4); pip install aiohttp huggingface_hub; then snapshot_download of HF_TARGET_REPO (and HF_DRAFT_REPO for dflash2) into MODEL_DIR/DRAFT_DIR. Requires the MiaAI-Lab exllamav3 fork (nvfp4/fp8 KV and DFlash2 are fork-only). Config via .env (copied from .env.example).
# Full instructions: https://github.com/MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw/tree/1242187390f780dba907e30659f1639fd0b491c8

2 Install

git clone https://github.com/MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw && cd Qwen3.8-27B-DFlash2-EXL3-5.0bpw && git checkout 1242187390f780dba907e30659f1639fd0b491c8

3 Start the server

CACHE_QUANT=nvfp4 CONTEXT_SIZE=262144 CPU_CACHE_GB=0 DRAFT=dflash2 DRAFT_DIR=models/Qwen3.8-27B-DFlash2-EXL3-5.0bpw GPU_MEM_GB=110 HOST=0.0.0.0 MODEL_DIR=models/Qwen3.8-27B-EXL3-3.5bpw PORT=8888 TORCH_CUDA_ARCH_LIST='12.0;12.1' TORCH_INDEX_URL=https://download.pytorch.org/whl/cu130 .venv/bin/python -u tools/serve_openai.py --model models/Qwen3.8-27B-EXL3-3.5bpw --host 0.0.0.0 --port 8888 --cache_size 262144 --grid_size 110 --cache_quant nvfp4 --draft_model models/Qwen3.8-27B-DFlash2-EXL3-5.0bpw

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw@1242187390f7, where it reports 47.5 tok/s. Our six checks have not run on it yet.

Details

Weights: inside the image
Image: none: a program on the host
Source: MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw @ 1242187390f780db ›
Launch: exllamav3-qwen3.8-27b-exl3-3.5bpw-256k ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/qwen3.8-27b.exllamav3.256k.json ›
Model released: Aug 5, 2026
Tested: not yet by us; reported by miaai-lab