Ling-3.0-Flash

· sglang · 256K context · DGX Spark GB10 · 128 GB

inclusionAI's Ling 3.0 Flash mixture-of-experts.

83 tok/sdecode

– tok/sprefill

256Kcontext window

Reported by MiaAI-LabAug 25, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/Ling-3.0-Flash-SGLang-DSpark-DGX-Spark@ca840cb8d032 › The server then answers on http://localhost:8000/v1.

1 Set up first

# ./start.sh --download-only caches weights + draft in $HF_HOME (hf/huggingface-cli or docker fallback); start.sh writes /tmp/ling-bootstrap.sh on the host and mounts it as the entrypoint (runs `sglang serve` or `python -m sglang.launch_server` with the args).
# Full instructions: https://github.com/MiaAI-Lab/Ling-3.0-Flash-SGLang-DSpark-DGX-Spark/tree/ca840cb8d032353e24648aeee06312b0938348f6

2 Download the weights

hf download inclusionAI/Ling-3.0-flash-int4 \
  --revision 7a27e9eb8179b2c2eb71eb214f0dab14ec6a63f2

3 Start the server

docker run --rm \
  --gpus all \
  -p 8000:8888 \
  --user root \
  --network host \
  --ipc host \
  --gpus all \
  --ulimit memlock=-1:-1 \
  --cap-add=IPC_LOCK \
  --workdir /workspace \
  -v ${BOOTSTRAP_SCRIPT}:/bootstrap.sh:ro \
  -v ${HF_HOME}:/root/.cache/huggingface \
  -v ${FLASHINFER_CACHE_DIR}:/root/.cache/flashinfer \
  -v ${WORK_DIR}:/workspace \
  --shm-size 32g \
  -e FLASHINFER_DISABLE_VERSION_CHECK=1 \
  -e GITHUB_TOKEN='${GITHUB_TOKEN:-}' \
  -e HF_HOME=/root/.cache/huggingface \
  -e HF_TOKEN='${HF_TOKEN:-}' \
  -e SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 \
  -e SGLANG_ENABLE_SPEC_V2=1 \
  -e SGLANG_JIT_DEEPGEMM_PRECOMPILE=1 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint /bootstrap.sh \
  lmsysorg/sglang:dev-Ling-3.0-flash@sha256:880d81d428e73aaad8eb944e33556951d17f9deaeba661803da5d391052840b6 \
  --model-path "${MODEL_PATH_IN_CONTAINER}" \
  --trust-remote-code \
  --nnodes 1 \
  --dist-init-addr 127.0.0.1:2345 \
  --host "${HOST}" \
  --port "${PORT}" \
  --tp-size 1 \
  --ep-size 1 \
  --random-seed 308534008 \
  --max-running-requests '${MAX_RUNNING_REQUESTS:-6}' \
  --max-mamba-cache-size '${MAX_MAMBA_CACHE_SIZE:-32}' \
  --chunked-prefill-size 8192 \
  --allow-auto-truncate \
  --context-length "${CTX}" \
  --mem-fraction-static '${MEM_FRACTION_STATIC:-0.75}' \
  --tool-call-parser ling3 \
  --reasoning-parser ling3 \
  --enable-fp32-lm-head \
  --disable-shared-experts-fusion \
  --kv-cache-dtype '${KV_CACHE_DTYPE:-fp8_e4m3}' \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path "${DSPARK_PATH_IN_CONTAINER}" \
  --enable-linear-replayssm-spec \
  --linear-replayssm-cache-len "${REPLAYSSM_CACHE_LEN}"

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/Ling-3.0-Flash-SGLang-DSpark-DGX-Spark@ca840cb8d032, where it reports 83 tok/s. Our six checks have not run on it yet.

Details

Weights: inclusionAI/Ling-3.0-flash-int4 @ 7a27e9eb81 ›
Image: lmsysorg/sglang:dev-Ling-3.0-flash@sha256:880d81d428e73aaad8eb944e33556951d17f9deaeba661803da5d391052840b6
Source: MiaAI-Lab/Ling-3.0-Flash-SGLang-DSpark-DGX-Spark @ ca840cb8d ›
Launch: sglang-ling-3.0-flash-int4-256k ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/ling-3.0-flash.sglang.256k.json ›
Model released: Aug 2, 2026
Tested: not yet by us; reported by miaai-lab