Qwen3.8-Flash-Next

GGUF IQ3_S · strata · 128K context · RTX 3090 · 24 GB

Qwen3.8's large mixture-of-experts model: far more knowledge than the 27B, with few active parameters, so it stays fast.

75 tok/sdecode

2,174 tok/sprefill

128Kcontext window

Tested on this cardOct 11, 2026

Run it

Follow the steps below to download and run this recipe. Omarchy Local AI lists the recipes it can launch on your hardware. The server then answers on http://localhost:8080/v1.

1 Set up first

# NEEDS: one 24 GB NVIDIA card (sm_86 build), 64 GB RAM or more (Strata's setup asks 62 GB for IQ3_S), ~92 GB on a local NVMe SSD (83.6 GB GGUF + 1.5 GB pack + 6.9 GB MTP layer). Low-RAM mode off: at start Strata copies the experts its VRAM cache does not hold into a host arena (48.8 GiB anonymous memory during decode); the GGUF stays mapped through the OS file cache (~55 GiB once warm, which MemAvailable counts as available). Strata serves one request at a time. KV: int8, the first 32,768 tokens on the GPU and the rest streamed from RAM (--kv-resident 32768; ~1.8 GB of RAM at 128K). Measured on an RTX 3090 (PCIe 4.0 x8, headless, Ryzen 9 5950X, 128 GB DDR4, NVMe): VRAM 23.7 GiB peak, ~15 CPU cores busy during decode. host_ram_gb is the largest MemAvailable drop from before the start to the end of the lab gates.
# Full instructions: https://github.com/Niko1221/Strata/tree/d5ea7133741e67743c0e886bb426c0ce8d69cf6c

2 Install the engine

# git clone https://github.com/Niko1221/Strata && git -C Strata checkout d5ea7133741e67743c0e886bb426c0ce8d69cf6c && cd Strata && ./setup.sh --family qwen --model IQ3_S --context 131072 --vision no --kv int8 --low-ram off --kv-streaming on --build --no-start --yes (compiles engine 0.1.40.3 for sm_86; our build of this commit, from Strata's own Dockerfile with CUDA 13.0: engine/strata sha256 7f532b61736a7ba1e3042c59eecbf7471e4b2c1aebb33f70e3f38be400189eaf; setup downloads the pinned GGUF shards, builds the pack and fetches the MTP layer, and writes strata-iq3_s.json)

3 Download the weights

hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00001-of-00002.gguf IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00002-of-00002.gguf \
  --revision ed59f92082b1e93c0e96d60a8b11aab089b52f09 \
  --local-dir ../Strata-data/models/IQ3_S

4 Start the server

.venv/bin/python \
  serve/server.py \
  --engine strata \
  --config strata-iq3_s.json \
  --port 8080

What it passed

✓ loadLoads and serves within an hour

✓ chatAnswers a plain question and stops on its own

✓ reasoningThinks separately and gets 17 × 23 right

✓ toolsCalls a tool with the right arguments and uses the result

✓ contextRecalls a code buried in a prompt that fills 85% of the window

✓ speedDecodes at 15 tok/s or more

Passed all six checks on a real RTX 3090 on Oct 11, 2026.

Details

Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF @ ed59f92082 ›
Image: none: a program on the host
Source: Niko1221/Strata @ d5ea7133741e67743c0e886bb426c0ce8d69cf6c ›
Launch: strata-qwen3.8-flash-next-iq3-s-128k-rtx-3090-24gb ›
Recipe file: registry/recipes/nvidia/rtx-3090-24gb/qwen3.8-flash-next.strata.128k.json ›
Model released: Aug 24, 2026
Tested: owner, RTX 3090