Qwen3.8-27B

GGUF IQ3_XXS · llama.cpp · 200K context · Apple M4 · 24 GB

Alibaba's 27B dense model from August 2026, and the strongest model that fits one consumer card. It thinks before it answers, calls tools reliably and reads images, which makes it the default for coding agents and long projects.

7 tok/sdecode

47 tok/sprefill

200Kcontext window

Reported by WyrmKeepSep 25, 2026

Run it

This is the launch as WyrmKeep publishes it. Their repository has the full setup: https://github.com/0xSero/local-ai-registry/pull/109 › The server then answers on http://localhost:8080/v1.

1 Install the engine

curl -L -o llama-b11182-bin-macos-arm64.tar.gz https://github.com/ggml-org/llama.cpp/releases/download/b11182/llama-b11182-bin-macos-arm64.tar.gz
echo 'ca2000b4037c3a467ec0758538af7e6f2b4dc2904f593bbedf4b28a97b6c444a  llama-b11182-bin-macos-arm64.tar.gz' | shasum -a 256 -c
tar xzf llama-b11182-bin-macos-arm64.tar.gz

2 Download the weights

hf download unsloth/Qwen3.8-27B-GGUF Qwen3.8-27B-UD-IQ3_XXS.gguf mmproj-BF16.gguf \
  --revision 4ca720788d1e01f1bff70c033e0d0028fd02e502 \
  --local-dir models/unsloth-Qwen3.8-27B-GGUF-4ca72078/IQ3_XXS

3 Raise the GPU memory limit (resets on reboot)
```
sudo sysctl iogpu.wired_limit_mb=20480
```

4 Start the server

llama-b11182/llama-server \
  -m models/unsloth-Qwen3.8-27B-GGUF-4ca72078/IQ3_XXS/Qwen3.8-27B-UD-IQ3_XXS.gguf \
  --mmproj models/unsloth-Qwen3.8-27B-GGUF-4ca72078/IQ3_XXS/mmproj-BF16.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 1 \
  -c 204800 \
  -np 1 \
  -fit off \
  -fa on \
  -ctk q4_0 \
  -ctv q4_0 \
  -ctkd q4_0 \
  -ctvd q4_0 \
  -ngl 99 \
  -ctxcp 4 \
  -cram 0 \
  --image-min-tokens 1024 \
  --jinja \
  --alias qwen3.8-27b \
  --host 127.0.0.1 \
  --port 8080

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by WyrmKeep in https://github.com/0xSero/local-ai-registry/pull/109, where it reports 7.1 tok/s. Our six checks have not run on it yet.

Details

Weights: unsloth/Qwen3.8-27B-GGUF @ 4ca720788d ›
Image: none: a program on the host
Source: https://github.com/0xSero/local-ai-registry/pull/109 ›
Launch: llama.cpp-metal-qwen3.8-27b-ud-iq3-xxs-mtp-200k ›
Recipe file: registry/recipes/apple/apple-m4-24gb/qwen3.8-27b.llama.cpp.200k.json ›
Model released: Aug 5, 2026
Tested: not yet by us; reported by WyrmKeep