Sybil AI
‹ RTX 4060 Ti

Qwen3.8-27B

EXL3 2.5 bpw · exllamav3 · 172K context · RTX 4060 Ti · 16 GB

Alibaba's 27B dense model from August 2026, and the strongest model that fits one consumer card. It thinks before it answers, calls tools reliably and reads images, which makes it the default for coding agents and long projects.

– tok/sdecode
– tok/sprefill
172Kcontext window
Reported by MiaAI-LabSep 10, 2026
Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install@622c7965ed5e › The server then answers on http://localhost:8888/v1.

  1. 1 Set up first
    # Not Docker; Windows + Linux. linux/setup.sh (or windows\START-HERE.bat) builds .venv, installs torch from cu128 index (aarch64: cu130), installs upstream exllamav3 v1.4.4 (prebuilt wheel from turboderp-org/exllamav3 release matched by tools/wheels.py, else source build of git+https://github.com/turboderp-org/exllamav3.git@v1.4.4; kit refuses any other engine version), aiohttp, huggingface_hub; tools/profiles.py picks quant/context from VRAM and writes .env; tools/downloader.py fetches HF_TARGET_REPO@HF_REVISION. linux/start.sh then execs the server and starts DeepSeek Harness (npx, Node 22.19+) on 127.0.0.1:3080 unless UI=no. Requires Python 3.11+, driver 570+, CC 7.5+.
# Full instructions: https://github.com/MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install/tree/622c7965ed5e02a13b188c9ef21bb9857fd2ba28
  2. 2 Install
    git clone https://github.com/MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install && cd Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install && git checkout 622c7965ed5e02a13b188c9ef21bb9857fd2ba28
  3. 3 Start the server
    CACHE_QUANT=4 CONTEXT_SIZE=176128 CPU_CACHE_GB=0 DRAFT=mtp GPU_MEM_GB=14.7 HF_REVISION=2.50bpw HF_TARGET_REPO=turboderp/Qwen3.8-27B-exl3 HOST=0.0.0.0 MODEL_DIR=models/Qwen3.8-27B-EXL3-2.5bpw MODEL_ID=qwen3.8-27b-exl3-2.5bpw PORT=8888 PROFILE=2.5bpw-176k SIMPLEX_HARNESS_PORT=3080 UI=browser VISION=auto .venv/bin/python -u tools/serve_openai.py --model models/Qwen3.8-27B-EXL3-2.5bpw --model_id qwen3.8-27b-exl3-2.5bpw --host 0.0.0.0 --port 8888 --cache_size 176128 --grid_size 14.7 --cache_quant 4 --draft_model mtp --harness_port 3080
What it passed
– loadLoads and serves within an hour
– chatAnswers a plain question and stops on its own
– reasoningThinks separately and gets 17 × 23 right
– toolsCalls a tool with the right arguments and uses the result
– contextRecalls a code buried in a prompt that fills 85% of the window
– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install@622c7965ed5e. Our six checks have not run on it yet.

Details
Weights
inside the image
Image
none: a program on the host
Source
MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install @ 6 ›
Launch
exllamav3-qwen3.8-27b-exl3-2.5bpw-172k-rtx-4060-ti-16gb ›
Recipe file
registry/recipes/nvidia/rtx-4060-ti-16gb/qwen3.8-27b.exllamav3.172k.json ›
Model released
Aug 5, 2026
Tested
not yet by us; reported by miaai-lab