MiMo-V2.6-Flash

· sglang · 1024K context · 2 × DGX Spark GB10 · 256 GB

Xiaomi's MiMo V2.6 Flash (RL build).

70 tok/sdecode

– tok/sprefill

1024Kcontext window

Reported by MiaAI-LabSep 22, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/MiMo-V2.6-Flash-2x-DGX-Sparks@201be3e743d2 › The server then answers on http://localhost:8000/v1.

1 Set up first

# RoCE over ConnectX-7 (RoCE v2 GID index auto-detected per node from HCA rocep1s0f0,rocep1s0f1); weights exported from head to worker via NFS docker volume (NFS_SHARE=1, files/nfs-share.sh); image built locally then docker save|ssh docker load to worker; patches/ bind-mounted at /opt/mimo26/patches (sitecustomize.py fixes MTP draft fused-qkv scale load at TP=2); boot.py/patches/state bind-mounted into container; optional host NCCL 2.30.7 overlay off by default (NCCL_OVERLAY_PIP=0)
# Full instructions: https://github.com/MiaAI-Lab/MiMo-V2.6-Flash-2x-DGX-Sparks/tree/201be3e743d215291feee09fe470822ec453c63d

2 Build the image

git clone https://github.com/MiaAI-Lab/MiMo-V2.6-Flash-2x-DGX-Sparks && cd MiMo-V2.6-Flash-2x-DGX-Sparks && git checkout 201be3e743d215291feee09fe470822ec453c63d
docker build -t local-ai/mimo-v2.6-flash.sglang.1024k.2x -f Dockerfile .

3 Download the weights

hf download XiaomiMiMo/MiMo-V2.6-Flash-RL \
  --revision 3b38d063180c3e4aed9691fdc735f3d10b266ee4 \
  --local-dir ~/models/MiMo-V2.6-Flash-RL-3b38d063

4 Start the server on each of the 2 machines (NODE_RANK 0 to 1)

docker run --rm \
  --gpus all \
  -p 8000:8888 \
  --network host \
  --ipc host \
  --privileged \
  --cap-add IPC_LOCK \
  --gpus all \
  --shm-size 32g \
  --ulimit memlock=-1:-1 \
  --ulimit stack=67108864 \
  --device /dev/infiniband:/dev/infiniband \
  --shm-size 32g \
  -e ATTENTION_BACKEND=triton \
  -e CHUNKED_PREFILL_SIZE=2048 \
  -e CONTEXT_LENGTH=1048576 \
  -e CUDA_DEVICE_ORDER=PCI_BUS_ID \
  -e CUDA_GRAPH_MAX_BS_DECODE=8 \
  -e DIST_INIT_ADDR='${HEAD_IP}:20000' \
  -e DP_SIZE=1 \
  -e ENABLE_CACHE_REPORT=1 \
  -e ENABLE_METRICS=1 \
  -e ENABLE_MTP=1 \
  -e EP_SIZE=2 \
  -e GLOO_SOCKET_IFNAME=enP7s7 \
  -e HOST=0.0.0.0 \
  -e HOST_IP="${NODE_IP}" \
  -e KV_CACHE_DTYPE=fp8_e4m3 \
  -e MAX_RUNNING_REQUESTS=8 \
  -e MAX_TOTAL_TOKENS=2299968 \
  -e MEM_FRACTION_STATIC=0.93 \
  -e MEM_GUARD_GIB=4 \
  -e MM_ATTENTION_BACKEND=triton_attn \
  -e MODEL_PATH=/models/MiMo-V2.6-Flash-RL \
  -e MOE_A2A_BACKEND=none \
  -e MOE_DENSE_TP_SIZE=1 \
  -e MOE_RUNNER_BACKEND=flashinfer_mxfp4 \
  -e NCCL_BUFFSIZE=1048576 \
  -e NCCL_CROSS_NIC=1 \
  -e NCCL_CUMEM_ENABLE=0 \
  -e NCCL_DEBUG=WARN \
  -e NCCL_IB_DISABLE=0 \
  -e NCCL_IB_GID_INDEX="${GID_INDEX}" \
  -e NCCL_IB_HCA=rocep1s0f0,rocep1s0f1 \
  -e NCCL_IB_MERGE_NICS=0 \
  -e NCCL_LL128_BUFFSIZE=262144 \
  -e NCCL_MAX_NCHANNELS=8 \
  -e NCCL_NET=IB \
  -e NCCL_P2P_DISABLE=1 \
  -e NCCL_PROTO='^LL128' \
  -e NCCL_SHM_DISABLE=1 \
  -e NCCL_SOCKET_IFNAME=enP7s7 \
  -e NNODES=2 \
  -e NODE_RANK="${NODE_RANK}" \
  -e OFFLOAD_MODE=off \
  -e PAGE_SIZE=64 \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False \
  -e REASONING_PARSER=mimo \
  -e SERVED_MODEL_NAME=MiMo-v2.6-Flash \
  -e SERVER_PORT=8888 \
  -e SGLANG_FLASHINFER_MOE_FUSED_FINALIZE=0 \
  -e SLEEP_ON_IDLE=1 \
  -e SPEC_ALGO=DFLASH \
  -e SPEC_DRAFT_MODEL='' \
  -e SPEC_DRAFT_TOKENS=8 \
  -e SPEC_STEPS=3 \
  -e SPEC_TOPK=1 \
  -e STATE_PATH=/state \
  -e SWA_FULL_TOKENS_RATIO=0.03 \
  -e TOOL_CALL_PARSER=mimo \
  -e TP_SIZE=2 \
  -e VLLM_HOST_IP="${NODE_IP}" \
  -e WATCHDOG_TIMEOUT=1800 \
  -e WEIGHT_LOADER_DROP_CACHE=1 \
  -v ~/models/MiMo-V2.6-Flash-RL-3b38d063:/models/MiMo-V2.6-Flash-RL:ro \
  --entrypoint python3 \
  local-ai/mimo-v2.6-flash.sglang.1024k.2x \
  -u /opt/mimo26/boot.py \
  run \
  --trust-remote-code \
  --model-path /models/MiMo-V2.6-Flash-RL \
  --served-model-name MiMo-v2.6-Flash \
  --enable-multimodal \
  --tp 2 \
  --ep 2 \
  --moe-a2a-backend none \
  --moe-dense-tp-size 1 \
  --mem-fraction-static 0.93 \
  --max-running-requests 8 \
  --chunked-prefill-size 2048 \
  --page-size 64 \
  --swa-full-tokens-ratio 0.03 \
  --context-length 1048576 \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --host 0.0.0.0 \
  --port 8888 \
  --speculative-algorithm DFLASH \
  --speculative-num-draft-tokens 8 \
  --speculative-draft-model-path /state/dflash \
  --attention-backend triton \
  --mm-attention-backend triton_attn \
  --moe-runner-backend flashinfer_mxfp4 \
  --cuda-graph-max-bs-decode 8 \
  --disable-prefill-cuda-graph \
  --weight-loader-drop-cache-after-load \
  --watchdog-timeout 1800 \
  --random-seed 0 \
  --enable-cache-report \
  --enable-metrics \
  --kv-cache-dtype fp8_e4m3 \
  --max-total-tokens 2299968 \
  --sleep-on-idle \
  --nnodes 2 \
  --node-rank "${NODE_RANK}" \
  --dist-init-addr '${HEAD_IP}:20000'

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/MiMo-V2.6-Flash-2x-DGX-Sparks@201be3e743d2, where it reports 69.8 tok/s. Our six checks have not run on it yet.

Details

Weights: XiaomiMiMo/MiMo-V2.6-Flash-RL @ 3b38d06318 ›
Image: built from MiaAI-Lab/MiMo-V2.6-Flash-2x-DGX-Sparks @ 201be3e743 ›
Source: MiaAI-Lab/MiMo-V2.6-Flash-2x-DGX-Sparks @ 201be3e743d215291f ›
Launch: sglang-mimo-v2.6-flash-other-1024k-2x ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/mimo-v2.6-flash.sglang.1024k.2x.json ›
Model released: Sep 21, 2026
Tested: not yet by us; reported by miaai-lab