Welp-35B-A3B

· llama.cpp · 256K context · RTX 4080 SUPER · 16 GB

Quaedra's 13.2 GB GGUF of Qwen3.6-35B-A3B: the routed experts quantized block-wise with GPTQ error feedback in standard llama.cpp formats, so the full 262k context fits on a 16 GB card.

154 tok/sdecode

2,028 tok/sprefill

256Kcontext window

Tested on this cardOct 9, 2026

Run it

Follow the steps below to download and run this recipe. Omarchy Local AI lists the recipes it can launch on your hardware. The server then answers on http://localhost:8000/v1.

1 Download the weights

hf download quaedra/Welp-35B-A3B-GGUF \
  --revision 3e4a3adbda8df07d2889978170d9955c2e59fb7a

2 Start the server

docker run --rm \
  --gpus all \
  -p 8000:8080 \
  --shm-size 16g \
  -e LLAMA_CACHE=/root/.cache/huggingface \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint /app/llama-server \
  ghcr.io/ggml-org/llama.cpp:server-cuda12-b11146@sha256:52192a23c7b258044463aebba78826cdff39a7393b713edc313324b343a7b836 \
  --hf-repo quaedra/Welp-35B-A3B-GGUF \
  --hf-file Welp-35B-A3B.gguf \
  --n-gpu-layers 99 \
  --flash-attn on \
  --ctx-size 262144 \
  --cache-type-k q4_0 \
  --cache-type-v q4_0 \
  --ubatch-size 256 \
  --parallel 1 \
  --jinja \
  --host 0.0.0.0 \
  --port 8080

What it passed

✓ loadLoads and serves within an hour

✓ chatAnswers a plain question and stops on its own

✓ reasoningThinks separately and gets 17 × 23 right

✓ toolsCalls a tool with the right arguments and uses the result

✓ contextRecalls a code buried in a prompt that fills 85% of the window

✓ speedDecodes at 15 tok/s or more

Passed all six checks on a real RTX 4080 Super on Oct 9, 2026.

Details

Weights: quaedra/Welp-35B-A3B-GGUF @ 3e4a3adbda ›
Image: ghcr.io/ggml-org/llama.cpp:server-cuda12-b11146@sha256:52192a23c7b258044463aebba78826cdff39a7393b713edc313324b343a7b836
Launch: llama.cpp-welp-35b-a3b-262k ›
Recipe file: registry/recipes/nvidia/rtx-4080-super-16gb/welp-35b-a3b.llama.cpp.256k.json ›
Model released: Oct 8, 2026
Tested: owner, RTX 4080 Super