DeepSeek-V4.1-Flash

· vllm · 256K context · 4 × H200 NVL in one machine · 564 GB

DeepSeek's V4.1 Flash, a large mixture-of-experts model.

225 tok/sdecode

17,035 tok/sprefill

256Kcontext window

Tested on this cardNov 2, 2026

Run it

Follow the steps below to download and run this recipe. Omarchy Local AI lists the recipes it can launch on your hardware. The server then answers on http://localhost:8000/v1.

1 Set up first

# Download deepseek-ai/DeepSeek-V4.1-Flash-1017 at the pinned revision (510 GB) to local NVMe and mount it at /models. The image reads the Engram tables from that directory at every decode step (DSV41_ENGRAM_DISK=1 in the image ENV), so keep it on NVMe, not network storage. The first start JIT-compiles FlashInfer and CuTe kernels for several minutes; mount /root/.cache to keep them. Clients should ask for usage once at the end (stream_options include_usage, the OpenAI default): with a reasoning parser, continuous_usage_stats makes vLLM recount the reasoning tokens of the whole output on every chunk, which cut 8k decode at 160 users from 6,810 to 1,290 tok/s here.
# Full instructions: https://github.com/sybil-solutions/local-ai-images/tree/vllm-dsv41-h200/vllm-dsv41-h200

2 Download the weights

hf download deepseek-ai/DeepSeek-V4.1-Flash-1017 \
  --revision 1bae0e1088993b9b82de08f59de5be5a34e42170 \
  --local-dir ~/models/DeepSeek-V4.1-Flash-1017-1bae0e10

3 Start the server

docker run --rm \
  --gpus all \
  -p 8000:8000 \
  --shm-size 16g \
  -v ~/models/DeepSeek-V4.1-Flash-1017-1bae0e10:/models:ro \
  --entrypoint vllm \
  ghcr.io/sybil-solutions/vllm-dsv41-h200@sha256:42b9c8311c3241e6f82dd85f92454a24c5ae17f353dcd427c99f0dea80554442 \
  serve \
  /models \
  --served-model-name deepseek-v4.1-flash \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 4 \
  --tokenizer-mode deepseek_v41 \
  --reasoning-parser deepseek_v41 \
  --tool-call-parser deepseek_v41 \
  --enable-auto-tool-choice \
  --default-chat-template-kwargs '{"thinking":true}' \
  --engram-config '{"cpu_offload":true}' \
  --max-model-len 262144 \
  --max-num-seqs 256 \
  --max-num-batched-tokens 16384 \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --enable-prompt-tokens-details \
  --gpu-memory-utilization 0.985 \
  --kv-cache-memory-bytes 41725000000 \
  --speculative-config '{"method":"dspark","num_speculative_tokens":5,"draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":true}' \
  --safetensors-load-strategy lazy \
  --no-disable-custom-all-reduce \
  --linear-backend humming \
  --moe-backend marlin \
  --additional-config '{"decode_backend":"latency_specialized"}' \
  --max-logprobs 128

What it passed

✓ loadLoads and serves within an hour

✓ chatAnswers a plain question and stops on its own

✓ reasoningThinks separately and gets 17 × 23 right

✓ toolsCalls a tool with the right arguments and uses the result

✓ contextRecalls a code buried in a prompt that fills 85% of the window

✓ speedDecodes at 15 tok/s or more

Passed all six checks on a real H200 NVL on Nov 2, 2026.

Details

Weights: deepseek-ai/DeepSeek-V4.1-Flash-1017 @ 1bae0e1088 ›
Image: ghcr.io/sybil-solutions/vllm-dsv41-h200@sha256:42b9c8311c3241e6f82dd85f92454a24c5ae17f353dcd427c99f0dea80554442
Source: sybil-solutions/local-ai-images @ vllm-dsv41-h200/vllm-dsv41 ›
Launch: vllm-dsv41-h200-deepseek-v4.1-flash-1017-256k-tp4 ›
Recipe file: registry/recipes/nvidia/h200-nvl-141gb/deepseek-v4.1-flash.vllm.256k.4x.json ›
Model released: Sep 10, 2026
Tested: owner, H200 NVL