Qwen3.8-27B

EXL3 4 bpw · exllamav3 · 128K context · Radeon RX 7900 XTX · 24 GB

Alibaba's 27B dense model from August 2026, and the strongest model that fits one consumer card. It thinks before it answers, calls tools reliably and reads images, which makes it the default for coding agents and long projects.

83 tok/sdecode

937 tok/sprefill

128Kcontext window

Tested on this cardOct 20, 2026

Run it

Follow the steps below to download and run this recipe. Omarchy Local AI lists the recipes it can launch on your hardware. The server then answers on http://localhost:5000/v1.

1 Set up first

# Not Docker; Linux on an RX 7900 XTX (gfx1100). setup/install.sh clones TabbyAPI at e086a65 (unpatched) into a Python 3.12 uv venv and installs the measured package set (setup/requirements.lock.txt: torch 2.13.0+rocm10.0.0, rocm 10.0.0 and amd-torch-device-gfx1100 from AMD's whl-next indexes, the official exllamav3 1.6.3+rocm10.0 wheel); nothing compiles and no system ROCm is needed, only the amdgpu kernel driver. setup/download_model.sh fetches the pinned weights; serve/run.sh starts TabbyAPI with serve/config.yml: 131072 context, cache_mode 8,8, max_batch_size 1, MTP draft with draft_num_tokens 3, reasoning, tool_format qwen3_coder, vision with vision_offload (encoder weights in system RAM). HIP_VISIBLE_DEVICES selects the discrete card over a Ryzen iGPU. Peaks around 21 GiB of the card's 24 GiB with the desktop on the same card.
# Full instructions: https://github.com/kamm3r/exl3-rdna3-recipe/tree/1fb72bdafc8bcf47170c3277d39494eef01c683b

2 Install the engine

git clone https://github.com/kamm3r/exl3-rdna3-recipe && cd exl3-rdna3-recipe && git checkout 1fb72bdafc8bcf47170c3277d39494eef01c683b

3 Download the weights

hf download turboderp/Qwen3.8-27B-exl3 \
  --revision 4acd9ad5af224ec9e8815a54d71a033f378a7e9d \
  --local-dir models/Qwen3.8-27B-exl3-SC4.0

4 Write the launcher

mkdir -p serve
cat > serve/config.yml <<'EOF'
# Qwen3.8-27B EXL3 4.0 bpw on one RX 7900 XTX (24 GB): 128K context, Q8 KV cache, MTP draft, vision.
# Paths are relative to tabbyAPI/ (serve/run.sh starts the server from there).
network:
  host: 127.0.0.1
  port: 5000
  disable_auth: true
  allowed_origins: []

logging:
  log_prompt: false
  log_requests: false

model:
  model_dir: ../models
  model_name: Qwen3.8-27B-exl3-SC4.0
  backend: exllamav3
  max_seq_len: 131072
  cache_size: 131072
  cache_mode: 8,8
  max_batch_size: 1
  chunk_size: 2048
  vision: true
  vision_offload: true
  reasoning: true
  tool_format: qwen3_coder

draft_model:
  draft_mode: mtp
  # The model has one MTP layer; drafting 3 tokens instead of the default 4 measured ~9% faster decode here.
  draft_num_tokens: 3
EOF

5 Start the server
```
HIP_VISIBLE_DEVICES=0 \
  serve/run.sh
```

What it passed

✓ loadLoads and serves within an hour

✓ chatAnswers a plain question and stops on its own

✓ reasoningThinks separately and gets 17 × 23 right

✓ toolsCalls a tool with the right arguments and uses the result

✓ contextRecalls a code buried in a prompt that fills 85% of the window

✓ speedDecodes at 15 tok/s or more

Passed all six checks on a real AMD Radeon RX 7900 XTX 24GB (owner: Nobara 44, kernel 7.2, ROCm 10.0 from pip, gfx1100) on Oct 20, 2026.

Details

Weights: turboderp/Qwen3.8-27B-exl3 @ 4acd9ad5af ›
Image: none: a program on the host
Source: kamm3r/exl3-rdna3-recipe @ 1fb72bdafc8bcf47170c3277d39494eef ›
Launch: exllamav3-rocm-qwen3.8-27b-exl3-4.0bpw-mtp-vision-128k-rx-7900-xtx-24gb ›
Recipe file: registry/recipes/amd/rx-7900-xtx-24gb/qwen3.8-27b.exllamav3.128k.json ›
Model released: Aug 5, 2026
Tested: owner, AMD Radeon RX 7900 XTX 24GB (owner: Nobara 44, kernel 7.2, ROCm 10.0 from pip, gfx1100)