Qwen3.8-27B

EXL3 3.5 bpw · exllamav3 · 195K context · RTX 3090 · 24 GB

Alibaba's 27B dense model from August 2026, and the strongest model that fits one consumer card. It thinks before it answers, calls tools reliably and reads images, which makes it the default for coding agents and long projects.

– tok/sdecode

– tok/sprefill

195Kcontext window

Reported by MiaAI-LabSep 1, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw@1242187390f7 › The server then answers on http://localhost:8888/v1.

1 Set up first

# Not Docker. start.sh self-bootstraps: python3 -m venv .venv; pip install torch --extra-index-url ${TORCH_INDEX_URL:-https://download.pytorch.org/whl/cu130}; pip install --no-build-isolation ${EXL3_REPO:-git+https://github.com/MiaAI-Lab/exllamav3} (CUDA kernel compile, 5-20 min; TORCH_CUDA_ARCH_LIST=12.0;12.1 auto-set on aarch64; MAX_JOBS 8 or 4); pip install aiohttp huggingface_hub; then snapshot_download of HF_TARGET_REPO (and HF_DRAFT_REPO for dflash2) into MODEL_DIR/DRAFT_DIR. Requires the MiaAI-Lab exllamav3 fork (nvfp4/fp8 KV and DFlash2 are fork-only). Config via .env (copied from .env.example).
# Full instructions: https://github.com/MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw/tree/1242187390f780dba907e30659f1639fd0b491c8

2 Install

git clone https://github.com/MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw && cd Qwen3.8-27B-DFlash2-EXL3-5.0bpw && git checkout 1242187390f780dba907e30659f1639fd0b491c8

3 Start the server

CACHE_QUANT=4 CONTEXT_SIZE=200000 CPU_CACHE_GB=0 DRAFT=dflash2 DRAFT_DIR=models/Qwen3.8-27B-DFlash2-EXL3-5.0bpw GPU_MEM_GB=22 HOST=0.0.0.0 MODEL_DIR=models/Qwen3.8-27B-EXL3-3.5bpw PORT=8888 .venv/bin/python -u tools/serve_openai.py --model models/Qwen3.8-27B-EXL3-3.5bpw --host 0.0.0.0 --port 8888 --cache_size 200000 --grid_size 22 --cache_quant 4 --draft_model models/Qwen3.8-27B-DFlash2-EXL3-5.0bpw

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw@1242187390f7. Our six checks have not run on it yet.

Details

Weights: inside the image
Image: none: a program on the host
Source: MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw @ 1242187390f780db ›
Launch: exllamav3-qwen3.8-27b-exl3-3.5bpw-195k-rtx-3090-24gb ›
Recipe file: registry/recipes/nvidia/rtx-3090-24gb/qwen3.8-27b.exllamav3.195k.json ›
Model released: Aug 5, 2026
Tested: not yet by us; reported by miaai-lab