Qwen3.8-Flash-Next

· strata · 64K context · RTX 5070 · 12 GB

Qwen3.8's large mixture-of-experts model: far more knowledge than the 27B, with few active parameters, so it stays fast.

30 tok/sdecode

807 tok/sprefill

64Kcontext window

Tested on this cardOct 8, 2026

Run it

Follow the steps below to download and run this recipe. Omarchy Local AI lists the recipes it can launch on your hardware. The server then answers on http://localhost:8080/v1.

1 Set up first

# NEEDS: one 12 GB NVIDIA card (sm_120 build), 32 GB RAM, ~80 GB on a local NVMe SSD (55 GB GGUF + 25 GB pack + MTP layer). Strata sizes the page-locked expert set at start from MemAvailable minus 4 GiB (at most 19.11 GiB for this model) and reads the experts it could not pin from the GGUF through the OS file cache: measured 19.85 GiB pinned at 51.4 tok/s and 15.78 GiB pinned at 50.4 tok/s (Strata's own token counts; the lab's speed gate estimates tokens as chars/4 because Strata has no tokenize endpoint, so its tps reads ~30% low). MemAvailable dropped 18.6 GB from start through the gates (swap grew 7.5 GB as the kernel made room). VRAM: 11.7 GB peak; Strata warns only ~215 MiB stays free with the desktop on the same GPU.
# Full instructions: https://github.com/Niko1221/Strata/tree/82f46a8c8f475f001ad76d92f58f4a4f8ffb0253

2 Install the engine

# git clone https://github.com/Niko1221/Strata && git -C Strata checkout 82f46a8c8f475f001ad76d92f58f4a4f8ffb0253 && cd Strata && ./setup.sh --family coder --context 65536 --vision no --low-ram resident --build --no-start --yes (compiles engine 0.1.40 for sm_120; engine/strata sha256 d1c8fc79180c936d8c9e6f69271d4ccd6a77104f49a7094869329bd04baba387; setup downloads the pinned GGUF shards, builds the pack and fetches the MTP layer, and writes strata-coder-iq1_m.json)

3 Download the weights

hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00002-of-00002.gguf \
  --revision 5348543e0147355ac9cbcb031184a3546350988e \
  --local-dir ../Strata-data/models/coder-IQ1_M

4 Start the server

.venv/bin/python \
  serve/server.py \
  --engine strata \
  --config strata-coder-iq1_m.json \
  --port 8080

What it passed

✓ loadLoads and serves within an hour

✓ chatAnswers a plain question and stops on its own

✓ reasoningThinks separately and gets 17 × 23 right

✓ toolsCalls a tool with the right arguments and uses the result

✓ contextRecalls a code buried in a prompt that fills 85% of the window

✓ speedDecodes at 15 tok/s or more

Passed all six checks on a real RTX 5070 on Oct 8, 2026.

Details

Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF @ 5348543e01 ›
Image: none: a program on the host
Source: Niko1221/Strata @ 82f46a8c8f475f001ad76d92f58f4a4f8ffb0253 ›
Launch: strata-qwen3.8-flash-next-coder-iq1-m-resident-64k-rtx-5070-12gb ›
Recipe file: registry/recipes/nvidia/rtx-5070-12gb/qwen3.8-flash-next.strata.64k.json ›
Model released: Aug 24, 2026
Tested: owner, RTX 5070