Step-3.7-Flash

NVFP4 · vllm · 256K context · 2 × DGX Spark GB10 · 256 GB

StepFun's Step 3.7 Flash mixture-of-experts model.

32 tok/sdecode

– tok/sprefill

256Kcontext window

Reported by MiaAI-LabJun 20, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/Dual-DGX-Spark-Step-3.7-Flash-NVFP4@11b9a4187a36 › The server then answers on http://localhost:8000/v1.

1 Set up first

# cp config.env.example config.env and set HEAD_IP/WORKER_IP (example link-local 169.254.x), ETH_IF, IB_IF. build-image.sh, copy-image-to-worker.sh, download-model.sh (huggingface-cli download on both nodes), then graft-mtp.sh, which runs lib/graft_mtp.py in the image on both nodes: it downloads the MTP shard(s) from stepfun-ai/Step-3.7-Flash (BF16), writes model-mtp.safetensors into the NVFP4 snapshot, updates the index, and extends the per-layer config lists. Containers start with --entrypoint= and 'sleep infinity'; the generated launch script is docker cp'd to /workspace/launch.sh and run with docker exec -d (worker first, head 5 s later). Passwordless SSH head->worker.
# Full instructions: https://github.com/MiaAI-Lab/Dual-DGX-Spark-Step-3.7-Flash-NVFP4/tree/11b9a4187a363bcc993ccda7d38edc02c7624dfd

2 Build the image

git clone https://github.com/MiaAI-Lab/Dual-DGX-Spark-Step-3.7-Flash-NVFP4 && cd Dual-DGX-Spark-Step-3.7-Flash-NVFP4 && git checkout 11b9a4187a363bcc993ccda7d38edc02c7624dfd
docker build -t local-ai/step-3.7-flash.vllm.256k.2x -f Dockerfile.stepfun37-workspace .

3 Download the weights

hf download stepfun-ai/Step-3.7-Flash-NVFP4 \
  --revision 4275532ffd9a9496ff36b7a2dc4a9db1048da438

4 Start the server on each of the 2 machines (NODE_RANK 0 to 1)

docker run --rm \
  --gpus all \
  -p 8000:8888 \
  --gpus all \
  --privileged \
  --ipc=host \
  --network host \
  --entrypoint= \
  -e NCCL_IB_GID_INDEX=0 \
  -e NCCL_IB_HCA=rocep1s0f1,roceP2p1s0f1 \
  -e NCCL_IGNORE_CPU_AFFINITY=1 \
  -e NCCL_SOCKET_IFNAME=enp1s0f1np1 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint bash \
  local-ai/step-3.7-flash.vllm.256k.2x \
  /workspace/launch.sh \
  serve \
  stepfun-ai/Step-3.7-Flash-NVFP4 \
  --served-model-name stepfun-ai/Step-3.7-Flash-NVFP4 \
  --trust-remote-code \
  --host 0.0.0.0 \
  --port 8888 \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --max-num-batched-tokens 8192 \
  --max-num-seqs 8 \
  --gpu-memory-utilization 0.85 \
  --quantization modelopt \
  --kv-cache-dtype fp8 \
  --disable-cascade-attn \
  --disable-custom-all-reduce \
  --no-enable-flashinfer-autotune \
  --enable-auto-tool-choice \
  --reasoning-parser step3p5 \
  --tool-call-parser step3p5 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --nnodes 2 \
  --node-rank "${NODE_RANK}" \
  --master-addr "${HEAD_IP}" \
  --master-port 29501 \
  '${HEADLESS_ON_WORKER:+--headless}'

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/Dual-DGX-Spark-Step-3.7-Flash-NVFP4@11b9a4187a36, where it reports 31.5 tok/s. Our six checks have not run on it yet.

Details

Weights: stepfun-ai/Step-3.7-Flash-NVFP4 @ 4275532ffd ›
Image: built from MiaAI-Lab/Dual-DGX-Spark-Step-3.7-Flash-NVFP4 @ 11b9a4187a ›
Source: MiaAI-Lab/Dual-DGX-Spark-Step-3.7-Flash-NVFP4 @ 11b9a4187a36 ›
Launch: vllm-step-3.7-flash-nvfp4-256k-2x ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/step-3.7-flash.vllm.256k.2x.json ›
Model released: May 23, 2026
Tested: not yet by us; reported by miaai-lab