Hy3-295B

NVFP4 · vllm · 128K context · 2 × DGX Spark GB10 · 256 GB

Tencent's Hunyuan 3 (Hy3) 295B mixture-of-experts.

– tok/sdecode

– tok/sprefill

128Kcontext window

Reported by MiaAI-LabJul 7, 2026

Run it

This is the launch as MiaAI-Lab publishes it. Their repository has the full setup: MiaAI-Lab/Hy3-Dual-DGX-Spark@d7510b52b234 › The server then answers on http://localhost:8000/v1.

1 Set up first

# sudo systemctl stop earlyoom on the head before every start (or move vllm/ray/python to --avoid in /etc/default/earlyoom). Head: python3 + huggingface_hub, downloads via snapshot_download and rsyncs the hub dir to the worker; image is docker save | ssh docker load to the worker. Containers run 'ray start ... --num-gpus=1 --object-store-memory=134217728 --disable-usage-stats --block' (head --head --port=26480, worker --address=10.0.0.1:26480); vLLM is launched with docker exec -d in hy3-head running /serve-tools-hy3.sh. Passwordless SSH head->worker.
# Full instructions: https://github.com/MiaAI-Lab/Hy3-Dual-DGX-Spark/tree/d7510b52b234569b66e6263c5ead982a4eb62756

2 Download the weights

hf download kodelow/Hy3-NVFP4-W4A16 \
  --revision 4f7e1f02f32f4662bdc47f1237f34995088867a9 \
  --local-dir ~/models/Hy3-NVFP4-W4A16-4f7e1f02

3 Start the server on each of the 2 machines (NODE_RANK 0 to 1)

docker run --rm \
  --gpus all \
  -p 8000:8888 \
  --network host \
  --ipc host \
  --privileged \
  --security-opt label=disable \
  --gpus all \
  --ulimit memlock=-1 \
  --ulimit stack=67108864 \
  -e CUDA_DEVICE_MAX_CONNECTIONS=32 \
  -e CUDA_DEVICE_ORDER=PCI_BUS_ID \
  -e GLOO_SOCKET_IFNAME=enp1s0f1np1 \
  -e MODEL_PATH='/models/snapshots/${MODEL_REVISION}' \
  -e NCCL_IB_DISABLE=0 \
  -e NCCL_IB_HCA=roceP2p1s0f0 \
  -e NCCL_MAX_NCHANNELS=4 \
  -e NCCL_MIN_NCHANNELS=4 \
  -e NCCL_SOCKET_IFNAME=enp1s0f1np1 \
  -e PORT=8888 \
  -e PYTHONUNBUFFERED=1 \
  -e RAY_memory_monitor_refresh_ms=0 \
  -e RAY_memory_usage_threshold=0.99 \
  -e SERVED_NAME=hy3 \
  -e VLLM_HOST_IP="${NODE_IP}" \
  -e VLLM_USE_B12X_FP8_GEMM=0 \
  -e VLLM_USE_B12X_MOE=0 \
  -e VLLM_USE_B12X_SPARSE_INDEXER=0 \
  -v ~/models/Hy3-NVFP4-W4A16-4f7e1f02:/models:ro \
  --entrypoint bash \
  ghcr.io/miaai-lab/hy3-dual-dgx-spark:vllm-probe-modded@sha256:9017669d03d30831973a44dd4ae330a5ddb710397b08eee3177fd077e1b5598f \
  -c 'exec > /tmp/hy3-serve.log 2>&1; export PYTHONUNBUFFERED=1; sed -i '\''s|"<think>"|"<think:opensource>"|g; s|"</think>"|"</think:opensource>"|g'\'' /usr/local/lib/python3.12/dist-packages/vllm/reasoning/hy_v3_reasoning_parser.py; sed -i '\''s|"<tool_calls>"|"<tool_calls:opensource>"|g; s|"</tool_calls>"|"</tool_calls:opensource>"|g; s|"<tool_call>"|"<tool_call:opensource>"|g; s|"</tool_call>"|"</tool_call:opensource>"|g; s|"<tool_sep>"|"<tool_sep:opensource>"|g; s|"<arg_key>"|"<arg_key:opensource>"|g; s|"</arg_key>"|"</arg_key:opensource>"|g; s|"<arg_value>"|"<arg_value:opensource>"|g; s|"</arg_value>"|"</arg_value:opensource>"|g'\'' /usr/local/lib/python3.12/dist-packages/vllm/tool_parsers/hy_v3_tool_parser.py; exec vllm serve ...' \
  serve \
  '/models/snapshots/${MODEL_REVISION}' \
  --served-model-name hy3 \
  --port 8888 \
  --tensor-parallel-size 2 \
  --distributed-executor-backend ray \
  --max-model-len 131072 \
  --max-num-seqs 6 \
  --kv-cache-dtype fp8_e4m3 \
  --gpu-memory-utilization 0.90 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}' \
  --tool-call-parser hy_v3 \
  --reasoning-parser hy_v3 \
  --enable-auto-tool-choice \
  --trust-remote-code \
  --enforce-eager

What it passed

– loadLoads and serves within an hour

– chatAnswers a plain question and stops on its own

– reasoningThinks separately and gets 17 × 23 right

– toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Published by MiaAI-Lab in MiaAI-Lab/Hy3-Dual-DGX-Spark@d7510b52b234. Our six checks have not run on it yet.

Details

Weights: kodelow/Hy3-NVFP4-W4A16 @ 4f7e1f02f3 ›
Image: ghcr.io/miaai-lab/hy3-dual-dgx-spark:vllm-probe-modded@sha256:9017669d03d30831973a44dd4ae330a5ddb710397b08eee3177fd077e1b5598f
Source: MiaAI-Lab/Hy3-Dual-DGX-Spark @ d7510b52b234569b66e6263c5ead9 ›
Launch: vllm-hy3-nvfp4-128k-2x ›
Recipe file: registry/recipes/nvidia/dgx-spark-gb10-128gb/hy3.vllm.128k.2x.json ›
Model released: Jul 2, 2026
Tested: not yet by us; reported by miaai-lab