Qwen3.8-27B

EXL3 3 bpw · exllamav3 · 128K context · Radeon RX 9070 XT · 16 GB

Alibaba's 27B dense model from August 2026, and the strongest model that fits one consumer card. It thinks before it answers, calls tools reliably and reads images, which makes it the default for coding agents and long projects.

116 tok/sdecode

967 tok/sprefill

128Kcontext window

Tested on this cardOct 7, 2026

Run it

Follow the steps below to download and run this recipe. Omarchy Local AI lists the recipes it can launch on your hardware. The server then answers on http://localhost:8096/v1.

1 Set up first

# Not Docker; Linux on an RX 9070 / 9070 XT (gfx1201). setup/ builds a venv with PyTorch 2.13 (ROCm 7.2 wheels), clones upstream exllamav3 dev at 0662fac and applies patches/exllamav3-rdna4.patch (gfx12 WMMA multi-row GEMV, HIP graphs off as upstream now defaults on ROCm (96838c9 backported), GQA split-decode attention, Gated DeltaNet accepted-input replay, 16/32-row batched verify), installs upstream TabbyAPI at 2fd6cc7 unpatched, and requantizes the bf16 DFlash2 draft to 3.0 bpw (setup/requant_draft.sh, a few minutes on the GPU). Needs system ROCm 7.x plus a host gcc; tested on Fedora 44 with ROCm 7.1.1. Serves 128K context shared by up to 3 concurrent requests; vision encoder kept in system RAM. Peaks around 16.1 GiB of the card's 16.
# Full instructions: https://github.com/parth-thakre/exl3-rdna4-recipe/tree/92901a8a5c5dc51d9dad172059266c7053e6fe9d

2 Install the engine

git clone https://github.com/parth-thakre/exl3-rdna4-recipe && cd exl3-rdna4-recipe && git checkout 92901a8a5c5dc51d9dad172059266c7053e6fe9d

3 Download the weights

hf download turboderp/Qwen3.8-27B-exl3 \
  --revision 6fe61ad620abfe97c5b49f9722c2bceeea4ccc28 \
  --local-dir models/Qwen3.8-27B-EXL3-3.0bpw

hf download incoai/Qwen3.8-27B-DFlash2 \
  --revision 015e795645c74b1a0eeef3b570031fb62e769bc5 \
  --local-dir models/Qwen3.8-27B-DFlash2-bf16

4 Start the server
```
serve/run_tabby.sh
```

What it passed

✓ loadLoads and serves within an hour

✓ chatAnswers a plain question and stops on its own

✓ reasoningThinks separately and gets 17 × 23 right

✓ toolsCalls a tool with the right arguments and uses the result

✓ contextRecalls a code buried in a prompt that fills 85% of the window

✓ speedDecodes at 15 tok/s or more

Passed all six checks on a real AMD Radeon RX 9070 XT 16GB (owner: Fedora 44, ROCm 7.1.1, gfx1201) on Oct 7, 2026.

Details

Weights: turboderp/Qwen3.8-27B-exl3 @ 6fe61ad620 ›
Image: none: a program on the host
Source: parth-thakre/exl3-rdna4-recipe @ 92901a8a5c5dc51d9dad1720592 ›
Launch: exllamav3-rdna4-qwen3.8-27b-exl3-3.0bpw-dflash2-128k-rx-9070-xt-16gb ›
Recipe file: registry/recipes/amd/rx-9070-xt-16gb/qwen3.8-27b.exllamav3.128k.json ›
Model released: Aug 5, 2026
Tested: owner, AMD Radeon RX 9070 XT 16GB (owner: Fedora 44, ROCm 7.1.1, gfx1201)