DeepSeek-V4.1-Flash

· sglang · 1024K context · 4 × DGX Spark GB10 · 512 GB

DeepSeek's V4.1 Flash, a large mixture-of-experts model.

88 tok/sdecode

5,925 tok/sprefill

1024Kcontext window

Reported by MiaAI-LabSep 25, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks@cad252b7d300 › The server then answers on http://localhost:8000/v1.

1 Set up first

# 4 Sparks need a RoCE switch or the switchless-ring (4 DACs, patched sparkring NCCL, NCCL_SWITCHLESS_RING_ONLY=1; docs/switchless-ring.md); cp .env.tp4.example .env.tp4, uncomment IMAGE=dsv41-4x-spark:canary-roce, BUILD_DOCKERFILE=Dockerfile.canary-roce and the last EXTRA_CONTAINER_ENV line, set EP_SIZE=1; ./start-tp4.sh build/share (NFSv4)/pack (Engram shards ~48 GiB/node)/serve; build fetches third-party trees at pinned commits
# Full instructions: https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks/tree/cad252b7d3000cf21d919764e70a6912b6d3b0b0

2 Build the image

git clone https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks && cd DeepSeek-v4.1-Flash-DGX-Sparks && git checkout cad252b7d3000cf21d919764e70a6912b6d3b0b0
docker build -t local-ai/deepseek-v4.1-flash.sglang.1024k.4x -f Dockerfile.canary-roce .

3 Download the weights

hf download deepseek-ai/DeepSeek-V4.1-Flash \
  --revision fb2764a5cf321eaa5070ca8f9e892818f477c16d \
  --local-dir ~/models/DeepSeek-V4.1-Flash-fb2764a5

4 Start the server on each of the 4 machines (NODE_RANK 0 to 3)

docker run --rm \
  --gpus all \
  -p 8000:8888 \
  --network host \
  --ipc host \
  --privileged \
  --cap-add IPC_LOCK \
  --gpus all \
  --ulimit memlock=-1:-1 \
  --ulimit stack=67108864 \
  --device /dev/infiniband:/dev/infiniband \
  --shm-size 32g \
  -e B12X_COMPILE_CACHE_DIR=/state/b12x-compile \
  -e B12X_ROCE_CACHE_DIR=/state/b12x-roce \
  -e B12X_ROCE_HCA=rocep1s0f0,roceP2p1s0f0 \
  -e CHUNKED_PREFILL_SIZE=4096 \
  -e CONTEXT_LENGTH=1048576 \
  -e CUDA_DEVICE_ORDER=PCI_BUS_ID \
  -e CUDA_GRAPH_MAX_BS_DECODE=16 \
  -e DIST_INIT_ADDR='${HEAD_IP}:20000' \
  -e DSPARK_BLOCK_SIZE=5 \
  -e DSPARK_SPS_TABLE=/state/dspark_sps.json \
  -e DSPARK_STS_TABLE=/state/dspark_sts.json \
  -e DSV41_AUTOTUNE_KEEP=1 \
  -e DSV41_BLOCK_VERIFY=1 \
  -e DSV41_CACHE_GIB=4 \
  -e DSV41_CACHE_WAYS=16 \
  -e DSV41_DRAFT_HEAD_FP8=1 \
  -e DSV41_DRAFT_HEAD_FP8_IMPL=tp4 \
  -e DSV41_DRAFT_MAIN_PROJ_SPLIT=1 \
  -e DSV41_DRAFT_TAU=0.7 \
  -e DSV41_ENGRAM_PREFETCH=1 \
  -e DSV41_ENGRAM_PREFETCH_CHECK=0 \
  -e DSV41_FAST_LOAD=1 \
  -e DSV41_FAST_LOAD_EP_SIZE=1 \
  -e DSV41_FAST_LOAD_INFLIGHT_GB=6 \
  -e DSV41_FAST_LOAD_N_EXPERTS=384 \
  -e DSV41_FAST_LOAD_TP_SLICE=auto \
  -e DSV41_FOLDED_FENCE=1 \
  -e DSV41_HC_FUSED=1 \
  -e DSV41_INDEXER_CHUNKED=1 \
  -e DSV41_IO_THREADS=96 \
  -e DSV41_L2_PREFETCH=1 \
  -e DSV41_LAUNCHER=tp4 \
  -e DSV41_LOOP_ABORT=1 \
  -e DSV41_LOOP_LINE_REPEATS=8 \
  -e DSV41_LOOP_NGRAM=32 \
  -e DSV41_LOOP_REPEATS=4 \
  -e DSV41_MAX_NEW_TOKENS=32768 \
  -e DSV41_MOE_B12X_NEXT=1 \
  -e DSV41_MOE_B12X_NEXT_DETERMINISTIC=1 \
  -e DSV41_MXFP8_BACKEND=b12x \
  -e DSV41_PACKED_DIR=/engram \
  -e DSV41_PREFILL_SP=1 \
  -e DSV41_PREFILL_SP_FP8=1 \
  -e DSV41_REPLICATED_SPLIT=wqkv_a,engram.wkv \
  -e DSV41_RESIDENT_SCALES=0 \
  -e DSV41_ROCE_GATHER=2097152 \
  -e DSV41_ROUTER_LIVE=1 \
  -e DSV41_SERIAL_WEIGHT_LOAD=0 \
  -e DSV41_SHARED_PAD_BUF_ROWS=256 \
  -e DSV41_SHARED_PAD_K=1 \
  -e DSV41_SOURCE=/models/DeepSeek-V4.1-Flash \
  -e DSV41_STATS_SECONDS=60 \
  -e DSV41_TP_PAD=0 \
  -e DSV41_VERIFY_CAP=conf:0.1 \
  -e DSV41_WO_A_W8=1 \
  -e DSV41_WO_A_W8_DRAFT=0 \
  -e DSV41_WO_A_W8_DROP=1 \
  -e DSV41_WO_A_W8_MID=1 \
  -e EP_SIZE=1 \
  -e EXTRA_SGLANG_ARGS='--fp8-gemm-backend flashinfer_cutlass --watchdog-timeout 1800 --min-free-slots-delay 1 --enable-deepseek-v4-fp4-indexer --enable-cache-report --sleep-on-idle' \
  -e GLOO_SOCKET_IFNAME=enP7s7 \
  -e HOST=0.0.0.0 \
  -e HOST_IP="${NODE_IP}" \
  -e LD_LIBRARY_PATH="${NCCL_CONTAINER_DIR}" \
  -e MAX_RUNNING_REQUESTS=16 \
  -e MAX_TOTAL_TOKENS=8000000 \
  -e MEM_FRACTION_STATIC=0.80 \
  -e MODEL_PATH=/models/DeepSeek-V4.1-Flash \
  -e NCCL_BUFFSIZE=1048576 \
  -e NCCL_CROSS_NIC=1 \
  -e NCCL_CUMEM_ENABLE=0 \
  -e NCCL_DEBUG=WARN \
  -e NCCL_DEBUG_SUBSYS=INIT,ENV \
  -e NCCL_IB_DISABLE=0 \
  -e NCCL_IB_GID_INDEX="${GID_INDEX}" \
  -e NCCL_IB_HCA=rocep1s0f0,rocep1s0f1 \
  -e NCCL_IB_MERGE_NICS=0 \
  -e NCCL_IB_SUBNET_AWARE_ROUTING=1 \
  -e NCCL_LL128_BUFFSIZE=262144 \
  -e NCCL_MAX_NCHANNELS=8 \
  -e NCCL_NET=IB \
  -e NCCL_P2P_DISABLE=1 \
  -e NCCL_PROTO='^LL128' \
  -e NCCL_SHM_DISABLE=1 \
  -e NCCL_SOCKET_IFNAME=enP7s7 \
  -e NNODES=4 \
  -e NODE_RANK="${NODE_RANK}" \
  -e OFFLOAD_MODE=nvme \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False \
  -e SERVED_MODEL_NAME=deepseek-v4.1-flash \
  -e SERVER_PORT=8888 \
  -e SGLANG_DSPARK_FOLDED_SAMPLING=2 \
  -e SGLANG_DSV41_REASONING_EFFORT=75 \
  -e SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE=0 \
  -e SGLANG_FLASHINFER_MOE_FUSED_FINALIZE=0 \
  -e SGLANG_ROCE_ALLREDUCE=1 \
  -e SGLANG_ROCE_MAX_SIZE=2097152 \
  -e SGLANG_RUST_BUILD_MODE=never \
  -e SKIP_PREPARE=1 \
  -e SKIP_VERIFY=1 \
  -e SPARK_PREFILL_TP_MIN_CONTEXT=32768 \
  -e SPARK_PREFILL_TP_MIN_ROWS=1024 \
  -e SPARK_PREFILL_TP_SPLIT=1 \
  -e SPEC_ALGO=DSPARK \
  -e STATE_PATH=/state \
  -e TP_SIZE=4 \
  -e VLLM_HOST_IP="${NODE_IP}" \
  -e WARMUP=1 \
  -v ~/models/DeepSeek-V4.1-Flash-fb2764a5:/models/DeepSeek-V4.1-Flash:ro \
  --entrypoint python3 \
  local-ai/deepseek-v4.1-flash.sglang.1024k.4x \
  -m sglang.launch_server \
  --model-path /models/DeepSeek-V4.1-Flash \
  --served-model-name deepseek-v4.1-flash \
  --trust-remote-code \
  --load-format safetensors \
  --tp 4 \
  --ep-size 1 \
  --attention-backend dsv4 \
  --moe-runner-backend flashinfer_mxfp4 \
  --mem-fraction-static 0.80 \
  --chunked-prefill-size 4096 \
  --context-length 1048576 \
  --max-running-requests 16 \
  --cuda-graph-max-bs-decode 16 \
  --random-seed 0 \
  --enable-decoder-swa-bounded-replay \
  --enable-cache-report \
  --tool-call-parser deepseekv41 \
  --reasoning-parser deepseek-v41 \
  --host 0.0.0.0 \
  --port 8888 \
  --speculative-algorithm DSPARK \
  --speculative-dspark-block-size 5 \
  --nnodes 4 \
  --node-rank "${NODE_RANK}" \
  --dist-init-addr '${HEAD_IP}:20000' \
  --max-total-tokens 8000000 \
  --fp8-gemm-backend flashinfer_cutlass \
  --watchdog-timeout 1800 \
  --min-free-slots-delay 1 \
  --enable-deepseek-v4-fp4-indexer \
  --enable-cache-report \
  --sleep-on-idle

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks@cad252b7d300, where it reports 87.7 tok/s. Our six checks have not run on it yet.

Details

Weights: deepseek-ai/DeepSeek-V4.1-Flash @ fb2764a5cf ›
Image: built from MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks @ cad252b7d3 ›
Source: MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks @ cad252b7d3000cf21 ›
Launch: sglang-deepseek-v4.1-flash-other-1024k-4x ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/deepseek-v4.1-flash.sglang.1024k.4x.json ›
Model released: Sep 10, 2026
Tested: not yet by us; reported by miaai-lab