Ornith-1.5-35B-A3B

NVFP4 · vllm · 256K context · DGX Spark GB10 · 128 GB

Ornith 1.5, a 35B mixture-of-experts with built-in MTP.

86 tok/sdecode

– tok/sprefill

256Kcontext window

Reported by MiaAI-LabAug 19, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/Ornith-1.5-35B-A3B-DGX-Spark@9fd7fe789623 › The server then answers on http://localhost:8000/v1.

1 Set up first

# HF token must exist at $HF_CACHE/token; start.sh runs `hf download ornith-ai/Ornith-1.5-35B-A3B-NVFP4` on the host; mounts $HF_CACHE -> /root/.cache/huggingface and $HOME/.cache/vllm -> /root/.cache/vllm; bind-mounts two b12x patches read-only: patches/kernel.py -> /usr/local/lib/python3.12/dist-packages/b12x/moe/_shared/kernels/w4a16/kernel.py and patches/route_pack.py -> .../w4a16/route_pack.py (fix MTP + CUDA graph capture crash).
# Full instructions: https://github.com/MiaAI-Lab/Ornith-1.5-35B-A3B-DGX-Spark/tree/9fd7fe7896230550fad1c773e85ad765292cfde5

2 Download the weights

hf download ornith-ai/Ornith-1.5-35B-A3B-NVFP4 \
  --revision 94e431d9cc47fa1986a7a1a4e9a80f7f118b03aa

3 Start the server

docker run --rm \
  --gpus all \
  -p 8000:8888 \
  --gpus all \
  --network host \
  --ipc=host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  eugr/spark-vllm-b12x:latest@sha256:c9e22735a5bb701c64b04ff9e08779d75d90abc045523fe0adf87b6092457c98 \
  vllm \
  serve \
  "${MODEL_IN_CONTAINER}" \
  --served-model-name ornith-1.5-35b-a3b-nvfp4 \
  --host "${HOST}" \
  --port "${PORT}" \
  --trust-remote-code \
  --enable-prefix-caching \
  --spec-method mtp \
  --spec-tokens 1 \
  --max-num-seqs 24 \
  --moe-backend b12x \
  --gpu-memory-utilization "${GPU_UTIL}" \
  --max-model-len 262144 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/Ornith-1.5-35B-A3B-DGX-Spark@9fd7fe789623, where it reports 86.3 tok/s. Our six checks have not run on it yet.

Details

Weights: ornith-ai/Ornith-1.5-35B-A3B-NVFP4 @ 94e431d9cc ›
Image: eugr/spark-vllm-b12x:latest@sha256:c9e22735a5bb701c64b04ff9e08779d75d90abc045523fe0adf87b6092457c98
Source: MiaAI-Lab/Ornith-1.5-35B-A3B-DGX-Spark @ 9fd7fe7896230550fad ›
Launch: vllm-ornith-1.5-35b-a3b-nvfp4-256k ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/ornith-1.5-35b-a3b.vllm.256k.json ›
Model released: Aug 18, 2026
Tested: not yet by us; reported by miaai-lab