vLLM EXL3 3 bpw
45.5 decode tok/s
625 prefill tok/s
375K context
Reported by MiaAI-Lab · Aug 24, 2026
Exact recipevllm · FP8 · 1024K context vllm · 320K context vllm · 977K context 56 decode tok/s
— prefill tok/s
1024K context
Reported by MiaAI-Lab · Sep 16, 2026
vLLM FP8
41.9 decode tok/s
— prefill tok/s
1024K context
Reported by MiaAI-Lab · Sep 16, 2026
Exact recipevllm · 256K context vllm · EXL3 2.9 bpw · 586K context 22.5 decode tok/s
1,939 prefill tok/s
256K context
Tested on this card · Oct 3, 2026
SGLang Format not recorded
51 decode tok/s
2,000 prefill tok/s
256K context
Reported by MiaAI-Lab · Sep 25, 2026
SGLang Format not recorded
87.7 decode tok/s
5,925 prefill tok/s
1024K context
Reported by MiaAI-Lab · Sep 25, 2026
vLLM Format not recorded
55.8 decode tok/s
463 prefill tok/s
256K context
Reported by toncao · Sep 30, 2026
vLLM Format not recorded
101.4 decode tok/s
462 prefill tok/s
256K context
Reported by toncao · Sep 30, 2026
vLLM NVFP4
29.8 decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Jul 5, 2026
vLLM NVFP4
— decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Jul 16, 2026
vLLM Format not recorded
43.1 decode tok/s
843 prefill tok/s
128K context
Reported by toncao · Sep 30, 2026
vLLM Format not recorded
63.2 decode tok/s
412 prefill tok/s
256K context
Reported by toncao · Sep 30, 2026
vLLM Format not recorded
49 decode tok/s
401 prefill tok/s
256K context
Reported by toncao · Sep 30, 2026
vLLM Format not recorded
22.2 decode tok/s
98 prefill tok/s
256K context
Reported by toncao · Sep 30, 2026
Exact recipevllm · NVFP4 · 230K context vllm · NVFP4 · 340K context vllm · NVFP4 · 372K context 25.5 decode tok/s
— prefill tok/s
230K context
Reported by MiaAI-Lab · Aug 3, 2026
Exact recipesglang · NVFP4 · 256K context vllm · NVFP4 · 256K context 30.7 decode tok/s
— prefill tok/s
256K context
Earlier check · Date not recorded
Exact recipesglang · NVFP4 · 1024K context tensorfold · EXL3 4 bpw · 1024K context vllm · NVFP4 · 256K context 59.1 decode tok/s
— prefill tok/s
1024K context
Earlier check · Date not recorded
vLLM NVFP4
— decode tok/s
— prefill tok/s
128K context
Reported by MiaAI-Lab · Jul 7, 2026
SGLang NVFP4
33.9 decode tok/s
— prefill tok/s
1024K context
Reported by MiaAI-Lab · Aug 2, 2026
vLLM NVFP4
76.7 decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Jul 22, 2026
vLLM FP8
21.1 decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Jul 4, 2026
SGLang Format not recorded
83 decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Aug 25, 2026
vLLM NVFP4
30 decode tok/s
— prefill tok/s
977K context
Reported by MiaAI-Lab · Jul 15, 2026
SGLang Format not recorded
69.8 decode tok/s
— prefill tok/s
1024K context
Reported by MiaAI-Lab · Sep 22, 2026
vLLM NVFP4
35 decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Aug 11, 2026
SGLang NVFP4
— decode tok/s
— prefill tok/s
1024K context
Reported by MiaAI-Lab · Aug 11, 2026
vLLM NVFP4
— decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Jul 8, 2026
vLLM NVFP4
86.3 decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Aug 19, 2026
vLLM NVFP4
45.6 decode tok/s
1,083 prefill tok/s
256K context
Reported by MiaAI-Lab · Jul 31, 2026
vLLM NVFP4
95.1 decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Jul 29, 2026
Exact recipeexllamav3 · EXL3 3.5 bpw · 256K context sglang · NVFP4 · 256K context vllm · NVFP4 · 977K context 47.5 decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Sep 1, 2026
vLLM Format not recorded
34.5 decode tok/s
582 prefill tok/s
256K context
Reported by toncao · Sep 30, 2026
vLLM FP8
29.7 decode tok/s
562 prefill tok/s
256K context
Reported by toncao · Sep 30, 2026
vLLM NVFP4
54.4 decode tok/s
2,962.2 prefill tok/s
256K context
Reported by MiaAI-Lab · Sep 16, 2026
vLLM NVFP4
31.5 decode tok/s
— prefill tok/s
256K context
Reported by MiaAI-Lab · Jun 20, 2026
vLLM Format not recorded
21.3 decode tok/s
888 prefill tok/s
256K context
Tested on this card · Oct 6, 2026
Speeds are the selected recipe’s recorded measurements, not estimates; “—” means unmeasured. Evidence distinguishes checks on this card, sibling-card checks, earlier checks and publisher reports.