Underdog Saluki 27B 1.0

· llama.cpp · 32K context · RTX 5080 · 16 GB

ConwayResearch's mixed 2-bit GGUF quantization of Qwen3.8-27B, using GSQ-RCO weights and an importance matrix for tool calling. This recipe serves text; the optional vision projector is not included.

73 tok/sdecode

1,553 tok/sprefill

32Kcontext window

Tested on this cardOct 9, 2026

Run it

Follow the steps below to download and run this recipe. Omarchy Local AI lists the recipes it can launch on your hardware. The server then answers on http://localhost:8000/v1.

1 Download the weights

hf download ConwayResearch/Underdog-Saluki-27B-1.0 Underdog-Saluki-27B-1.0-IQ2-mix.gguf \
  --revision 1336c0b5d74dfe6ad7f793577092f5effd9f97c6 \
  --local-dir ~/models/Underdog-Saluki-27B-1.0-1336c0b5

2 Start the server

docker run --rm \
  --gpus all \
  -p 8000:8080 \
  --shm-size 8g \
  -v ~/models/Underdog-Saluki-27B-1.0-1336c0b5:/models:ro \
  --entrypoint /app/llama-server \
  ghcr.io/ggml-org/llama.cpp:server-cuda12-b11146@sha256:52192a23c7b258044463aebba78826cdff39a7393b713edc313324b343a7b836 \
  --model /models/Underdog-Saluki-27B-1.0-IQ2-mix.gguf \
  --alias underdog-saluki-27b-1.0 \
  --ctx-size 32768 \
  --parallel 1 \
  --n-gpu-layers 999 \
  --fit off \
  --flash-attn on \
  --no-mmproj \
  --jinja \
  --reasoning-format deepseek \
  --metrics \
  --host 0.0.0.0 \
  --port 8080

What it passed

✓ loadLoads and serves within an hour

✓ chatAnswers a plain question and stops on its own

✓ reasoningThinks separately and gets 17 × 23 right

✓ toolsCalls a tool with the right arguments and uses the result

✓ contextRecalls a code buried in a prompt that fills 85% of the window

✓ speedDecodes at 15 tok/s or more

Passed all six checks on a real RTX 5080 on Oct 9, 2026.

Details

Weights: ConwayResearch/Underdog-Saluki-27B-1.0 @ 1336c0b5d7 ›
Image: ghcr.io/ggml-org/llama.cpp:server-cuda12-b11146@sha256:52192a23c7b258044463aebba78826cdff39a7393b713edc313324b343a7b836
Launch: llama.cpp-underdog-saluki-27b-iq2-mix-32k ›
Recipe file: registry/recipes/nvidia/rtx-5080-16gb/underdog-saluki-27b-1.0.llama.cpp.32k.json ›
Model released: Oct 8, 2026
Tested: owner, RTX 5080