DeepSeek-V4.1-Flash

· sglang · 256K context · 3 × DGX Spark GB10 · 384 GB

DeepSeek's V4.1 Flash, a large mixture-of-experts model.

51 tok/sdecode

2,000 tok/sprefill

256Kcontext window

Reported by MiaAI-LabSep 25, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks@cad252b7d300 › The server then answers on http://localhost:8000/v1.

1 Set up first

# 3-node ConnectX-7 RoCE triangle (NCCL_NET=IB, Gloo on LAN NIC); ./start.sh build on all nodes; ./start.sh share exports head checkpoint via NFSv4 (vllm-fn-nfs exporter, docker volume dsv41-weights on workers); ./start.sh pack writes per-rank Engram rows to local NVMe (~63 GiB/node, mounted at /engram); optional host NCCL 2.30.7 mounted from ~/nccl-2.30.7; passwordless ssh to workers
# Full instructions: https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks/tree/cad252b7d3000cf21d919764e70a6912b6d3b0b0

2 Build the image

git clone https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks && cd DeepSeek-v4.1-Flash-DGX-Sparks && git checkout cad252b7d3000cf21d919764e70a6912b6d3b0b0
docker build -t local-ai/deepseek-v4.1-flash.sglang.256k.3x -f Dockerfile .

3 Download the weights

hf download deepseek-ai/DeepSeek-V4.1-Flash \
  --revision fb2764a5cf321eaa5070ca8f9e892818f477c16d \
  --local-dir ~/models/DeepSeek-V4.1-Flash-fb2764a5

4 Start the server on each of the 3 machines (NODE_RANK 0 to 2)

docker run --rm \
  --gpus all \
  -p 8000:8888 \
  --network host \
  --ipc host \
  --privileged \
  --cap-add IPC_LOCK \
  --gpus all \
  --ulimit memlock=-1:-1 \
  --ulimit stack=67108864 \
  --device /dev/infiniband:/dev/infiniband \
  --shm-size 32g \
  -e CHUNKED_PREFILL_SIZE=768 \
  -e CONTEXT_LENGTH=262144 \
  -e CUDA_DEVICE_ORDER=PCI_BUS_ID \
  -e CUDA_GRAPH_MAX_BS_DECODE=4 \
  -e DIST_INIT_ADDR='${HEAD_IP}:20000' \
  -e DSPARK_BLOCK_SIZE=5 \
  -e DSPARK_SPS_TABLE=/state/dspark_sps.json \
  -e DSPARK_STS_TABLE=/state/dspark_sts.json \
  -e DSV41_AUTOTUNE_KEEP=1 \
  -e DSV41_BLOCK_VERIFY=1 \
  -e DSV41_CACHE_GIB=0 \
  -e DSV41_CACHE_WAYS=4 \
  -e DSV41_DRAFT_HEAD_FP8=1 \
  -e DSV41_DRAFT_TAU=1 \
  -e DSV41_ENGRAM_PREFETCH=1 \
  -e DSV41_ENGRAM_PREFETCH_CHECK=0 \
  -e DSV41_FOLDED_FENCE=0 \
  -e DSV41_IO_THREADS=96 \
  -e DSV41_LOOP_ABORT=1 \
  -e DSV41_LOOP_LINE_REPEATS=8 \
  -e DSV41_LOOP_NGRAM=32 \
  -e DSV41_LOOP_REPEATS=4 \
  -e DSV41_MAX_NEW_TOKENS=32768 \
  -e DSV41_MXFP8_BACKEND=b12x \
  -e DSV41_PACKED_DIR=/engram \
  -e DSV41_RESIDENT_SCALES=0 \
  -e DSV41_SERIAL_WEIGHT_LOAD=0 \
  -e DSV41_SOURCE=/models/DeepSeek-V4.1-Flash \
  -e DSV41_STATS_SECONDS=60 \
  -e DSV41_TP_PAD=1 \
  -e DSV41_VERIFY_CAP=conf:0.1 \
  -e DSV41_WO_A_W8=1 \
  -e DSV41_WO_A_W8_DRAFT=0 \
  -e DSV41_WO_A_W8_DROP=1 \
  -e DSV41_WO_A_W8_MID=1 \
  -e EP_SIZE=1 \
  -e EXTRA_SGLANG_ARGS='--fp8-gemm-backend flashinfer_cutlass --watchdog-timeout 1800 --sleep-on-idle' \
  -e GLOO_SOCKET_IFNAME=enP7s7 \
  -e HOST=0.0.0.0 \
  -e HOST_IP="${NODE_IP}" \
  -e LD_LIBRARY_PATH="${NCCL_CONTAINER_DIR}" \
  -e MAX_RUNNING_REQUESTS=4 \
  -e MAX_TOTAL_TOKENS=750000 \
  -e MEM_FRACTION_STATIC=0.95 \
  -e MODEL_PATH=/models/DeepSeek-V4.1-Flash \
  -e NCCL_BUFFSIZE=1048576 \
  -e NCCL_CROSS_NIC=1 \
  -e NCCL_CUMEM_ENABLE=0 \
  -e NCCL_DEBUG=WARN \
  -e NCCL_DEBUG_SUBSYS=INIT,ENV \
  -e NCCL_IB_DISABLE=0 \
  -e NCCL_IB_GID_INDEX="${GID_INDEX}" \
  -e NCCL_IB_HCA=rocep1s0f0,rocep1s0f1 \
  -e NCCL_IB_MERGE_NICS=0 \
  -e NCCL_IB_SUBNET_AWARE_ROUTING=1 \
  -e NCCL_LL128_BUFFSIZE=262144 \
  -e NCCL_MAX_NCHANNELS=8 \
  -e NCCL_NET=IB \
  -e NCCL_P2P_DISABLE=1 \
  -e NCCL_PROTO='^LL128' \
  -e NCCL_SHM_DISABLE=1 \
  -e NCCL_SOCKET_IFNAME=enP7s7 \
  -e NNODES=3 \
  -e NODE_RANK="${NODE_RANK}" \
  -e OFFLOAD_MODE=nvme \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False \
  -e SERVED_MODEL_NAME=deepseek-v4.1-flash \
  -e SERVER_PORT=8888 \
  -e SGLANG_DSV41_REASONING_EFFORT=75 \
  -e SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE=0 \
  -e SGLANG_FLASHINFER_MOE_FUSED_FINALIZE=0 \
  -e SKIP_PREPARE=1 \
  -e SKIP_VERIFY=1 \
  -e SPEC_ALGO=DSPARK \
  -e STATE_PATH=/state \
  -e TP_SIZE=3 \
  -e VLLM_HOST_IP="${NODE_IP}" \
  -e WARMUP=1 \
  -v ~/models/DeepSeek-V4.1-Flash-fb2764a5:/models/DeepSeek-V4.1-Flash:ro \
  --entrypoint python3 \
  local-ai/deepseek-v4.1-flash.sglang.256k.3x \
  -m sglang.launch_server \
  --model-path /models/DeepSeek-V4.1-Flash \
  --served-model-name deepseek-v4.1-flash \
  --trust-remote-code \
  --load-format safetensors \
  --tp 3 \
  --ep-size 1 \
  --attention-backend dsv4 \
  --moe-runner-backend flashinfer_mxfp4 \
  --mem-fraction-static 0.95 \
  --chunked-prefill-size 768 \
  --context-length 262144 \
  --max-running-requests 4 \
  --cuda-graph-max-bs-decode 4 \
  --random-seed 0 \
  --enable-decoder-swa-bounded-replay \
  --enable-cache-report \
  --tool-call-parser deepseekv41 \
  --reasoning-parser deepseek-v41 \
  --host 0.0.0.0 \
  --port 8888 \
  --speculative-algorithm DSPARK \
  --speculative-dspark-block-size 5 \
  --nnodes 3 \
  --node-rank "${NODE_RANK}" \
  --dist-init-addr '${HEAD_IP}:20000' \
  --max-total-tokens 750000 \
  --fp8-gemm-backend flashinfer_cutlass \
  --watchdog-timeout 1800 \
  --sleep-on-idle

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks@cad252b7d300, where it reports 51 tok/s. Our six checks have not run on it yet.

Details

Weights: deepseek-ai/DeepSeek-V4.1-Flash @ fb2764a5cf ›
Image: built from MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks @ cad252b7d3 ›
Source: MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks @ cad252b7d3000cf21 ›
Launch: sglang-deepseek-v4.1-flash-other-256k-3x ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/deepseek-v4.1-flash.sglang.256k.3x.json ›
Model released: Sep 10, 2026
Tested: not yet by us; reported by miaai-lab