Benchmarks
From CMP Wiki
More actions
Every throughput number the recipes lane found, one row per measurement, with the setup it was measured on and its source. Click a column header to sort. The recipe pages give the full conditions; this table is for comparing.
Read before comparing. Numbers differ by protocol and setup:
- Most recipe authors use the MiaAI-Lab protocol (400-token answers, temperature 0, thinking off) and report three columns, structured / code / prose. Ivan's lab bench uses 1,024-token answers and lists code / structured / prose; rows say which order applies when it differs.
- "1 user" is single-stream decode in tok/s. "Many users" is aggregate tok/s across all streams, with the user count in brackets.
- Prefill is prompt-reading tok/s; cold unless marked warm. Prompt length is in brackets where known.
- Power caps (140 to 250 W), link width (Gen2 x4 or x16), P2P and the host CPU all move the numbers. Speculative decoding (DFlash2, DSpark, MTP) is on in most rows; temperature-0 benchmarks inflate draft acceptance against real use.
- Ivan's lab rows are on his own 8-card box and are marked "lab notes". All other rows are one person's report unless the recipe page shows a confirmation.
- Card count is CMP 170HX 8 GB-variant cards unlocked to 64 GB unless stated.
| Model | Quant | Engine / recipe | Cards | Layout and links | Power | Workload | 1 user (tok/s) | Many users (tok/s) | Prefill (tok/s) | Source |
|---|---|---|---|---|---|---|---|---|---|---|
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB 1.8.0 | 4 | TP4, x16, P2P | 180 W | MiaAI-Lab 400 tok | 482.6 / 442.1 / 232.1 | 943.7 / 835.0 / 667.2 (8) | 3,161 | Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06 |
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB 1.8.0 | 4 | PP4, x16 | 180 W | MiaAI-Lab 400 tok | 232.4 / 209.9 / 100.1 | 664.1 / 608.8 / 469.2 (8) | 7,419 | Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06 |
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB | 4 | TP4, x16, P2P | real workflows | 350+ bursts, 270+ real | about 800 (agents) | Source: @cibernox, 2026-10-13 | ||
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB | 4 | TP4 | bs=1 | 317 code, 164 prose | Source: @turbine, 2026-10-27 | |||
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB | 4 | PP4, Gen2 x4 | 1M context | 90 to 140 | about 3,000 | Source: @mountains1605, 2026-10-04 | ||
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB | 4 | PP4, Gen2 x4 | real agent work | 340 (8) | Source: @mountains1605, 2026-10-08 | |||
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB early release | 4 | 768 tok, mixed, reasoning counted | 182.1 / 161.8 / 141.2 | 231.0 (2), 307.5 (4), 296.1 (8) | Source: @ikyo, 2026-10-06 | |||
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB 1.7 | 4 | PP4 | c=3, 1M ctx | 75+ prose (per stream) | Source: @1a2b3c0967, 2026-10-04 | |||
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB (recipe not named) | 4 | P2P on | 180 W cap | single concurrency | 200 to 400 | Source: @peter643, 2026-10-20 | ||
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB 1.6.0 | 4 | TP4, PLX, P2P off | 180 W | MiaAI-Lab | 396.0 structured | 798.6 (8) | Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @c133, 2026-10-20 | |
| GLM 5.3 Flash | W4A16 | GLM 5.3 Flash on 4x 8 GB 1.6.0, HBM equalised | 4 | TP4, PLX, P2P off | 150 W | MiaAI-Lab | 418.0 structured | 816.5 (8) | Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @c133, 2026-10-20 | |
| GLM 5.3 Flash | W4A16 abliterated | GLM 5.3 Flash on 4x 8 GB 1.8.0 engine | 4 | TP4, x16, NCCL only | governor 170 to 230 W | 1,024 tok, effort low | 252 / 235 / 139 (code / struct / prose) | 623 / 617 / 421 (8) | 1,700 (17k) | Source: @darkmaniac7, 2026-10-28 (lab notes) |
| GLM 5.3 Flash | W4A16 abliterated | GLM 5.3 Flash on 4x 8 GB 1.8.0 engine | 4 | TP2 x PP2 | 1,024 tok | 171 / 146 / 98 (code / struct / prose) | 567 / 503 / 383 (8) | 4,200 (17k) | Source: @darkmaniac7, 2026-10-28 (lab notes) | |
| GLM 5.3 Flash | W4A16 abliterated | GLM 5.3 Flash on 4x 8 GB 1.8.0 engine | 4 | TP1 x PP4 | 1,024 tok | 113 / 121 / 71 (code / struct / prose) | 499 / 515 / 376 (8) | 6,800 (17k) | Source: @darkmaniac7, 2026-10-28 (lab notes) | |
| GLM 5.3 Flash | W4A16 abliterated | Morrowmake engine | 8 | TP8 | 250 W | 1,024 tok | 95 / 105 / 56 (code / struct / prose) | 440 (8) | 554 | Source: @darkmaniac7, 2026-10-27 (lab notes) |
| GLM 5.3 Flash | W4A16 | two instances of GLM 5.3 Flash on 4x 8 GB 1.6 | 8 | 2x PP4 or 2x TP4 | 180 to 200 W | aggregate | about 1,600 | Source: @impactic, 2026-10-20 | ||
| GLM 5.3 Flash | W4A16 | 4 cards | 4 | TP4 | 130 | Source: @turbine, 2026-10-06 | ||||
| GLM 5.3 Flash | EXL3 3.05 bpw | GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 TP2) | 2 | TP2, NCCL P2P, x16 | 180 W | single request | 96.71 / 107.67 / 47.83; counting 129.38 | 67.25 (2) | 1,512 to 1,533 | Source: repo github.com/alexvnesta/glm53-flash-cmp170hx-tp by alexvnesta on GitHub, posted by @nestaa51, 2026-10-12 |
| GLM 5.3 Flash | EXL3 3.05 bpw | GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split) | 2 | layer split | easy / hard text | 96.2 / 34.2 | Source: repo github.com/alexvnesta/glm53-flash-cmp170hx-tp by alexvnesta on GitHub, posted by @nestaa51, 2026-10-12 | |||
| GLM 5.3 Flash | EXL3 3.05 bpw | GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split) | 2 | layer split, no DFlash | about 60 | Source: @68x0, 2026-10-20 | ||||
| GLM 5.3 Flash | EXL3 3.05 bpw | Flun repo scaled to 3 cards | 3 | TP | full context | about 60 | Source: @wingrimez_40467, 2026-10-20 | |||
| GLM 5.3 Flash | Maya-M GGUF | GLM 5.3 Flash on 2x 8 GB (Project Maya) v1.0.6 | 2 | Gen2 x4 | maya.sh --bench | 61.3 | 514 | Source: repo github.com/mw00/project-maya by mw00 on GitHub, posted by @pcsv., 2026-10-25 | ||
| GLM 5.3 Flash | Maya-S GGUF, 128K | GLM 5.3 Flash on 2x 8 GB (Project Maya) v1.1.15 | 2 (listed as 40 GB) | Gen2 x16 | maya.sh --bench | 54.3 | 831 | Source: repo github.com/mw00/project-maya by mw00 on GitHub, posted by @pcsv., 2026-10-25 | ||
| GLM 5.3 Flash | Maya-S24 GGUF, 32K | Project Maya v1.1.8 | 1 | Gen2 x16 | maya.sh --bench | 20.6 | 797 | Source: repo github.com/mw00/project-maya by mw00 on GitHub, posted by @pcsv., 2026-10-25 | ||
| GLM 5.3 Flash | EXL3 4-bit | ExLlamaV3, 2x CMP 64 GB + 40 GB card + RTX 3080 | 2 + 2 other | mixed | 768k ctx, 16-bit KV | about 70 | 3,500 to 3,600 (100k) | Source: @vektor_prime, 2026-10-14 | ||
| GLM 5.3 (full) | Int4-Int8Mix | GLM 5.3 on 8x 8 GB (vllm-170hx-glm5) | 10 | PP10 | 170 W | MAX_SEQS 8 | 48.7 | 108.4 (4), 174.1 (8) | 1,833; 2,351 aggregate (8) | Source: repo github.com/arvkmr/vllm-170hx-glm5 by arvkmr on GitHub, posted by @reala7096, 2026-10-08 |
| GLM 5.3 (full) | Int4-Int8Mix | GLM 5.3 on 8x 8 GB (Int4-Int8Mix PP8) | 8 | PP8, MTP 5 | 200 W | about 25 (about 10 at high context) | Source: @1a2b3c0967, 2026-10-07 | |||
| GLM 5.3 (full) | W4A16 | own setup (not published) | 8 | 64 code, 59 to 60 reasoning, 49 prose | 1,250 to 2,000 | Source: @impactic, 2026-10-23 | ||||
| GLM 5.3 (full) | own setup | 4 | 32 | Source: @mountains1605, 2026-10-20 | ||||||
| GLM 5.3 (full) | EXL3 3.05 bpw | ExLlamaV3, 4x CMP + RTX PRO 6000 | 4 + 1 other | PP | 30 | Source: @mountains1605, 2026-10-12 | ||||
| GLM 5.3 (full, 753B) | NVFP4 experts + INT8 (abliterated requant) | wtdcode/vllm-backport with the PP hand-off fix | 8 | TP2 x PP4, P2P, MTP | 110 to 130 W governor | 1,024 tok | 77 / 76 / 48 (code / struct / prose) | 172 / 189 / 146 (8) | 1,650 (17k) | Source: @darkmaniac7, 2026-10-28 (lab notes) |
| DeepSeek V4.1 Flash | EXL3 2.9 experts + FP8 | DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake) 1.0.0 | 4 | TP4, x16, P2P, Engram RAM | 180 W | MiaAI-Lab, up to | 319 / 287 / 162 | 765 / 729 / 432 (8) | 3,818 (2K), 3,808 (32K) | Source: repo github.com/Morrowmake/dsv41-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @lia_oc, 2026-10-24 |
| DeepSeek V4.1 Flash | EXL3 2.9 experts + FP8 | DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake) 1.0.0 | 4 | TP4, P2P off, PLX | 140 W | Curtis protocol | 122.8 / 90.3 / 62.4 (count / code / prose) | 168.7 (8) | 1,558.7 (128K) | Source: repo github.com/CurtisAccelerate/DeepSeekv41-Flash-1017-EXL3-CMP170HX by CurtisAccelerate on GitHub, posted by @anon33anon, 2026-10-20 |
| DeepSeek V4.1 Flash heretic | EXL3 2.9 experts + FP8 + heretic patch | DeepSeek V4.1 Flash heretic on 4x 8 GB | 4 | TP4, Engram SSD, miner co-resident | own bench | 136 / 158 / 121 | 261 / 323 / 249 (8) | 2,453 (17k warm) | Source: @darkmaniac7, 2026-10-29 (lab notes) | |
| DeepSeek V4.1 Flash | EXL3 experts + FP8 | DeepSeek V4.1 Flash on 4x 8 GB (GNmac) 1.0.0 | 4 | TP4, x16 | 180 W | MiaAI-Lab | 296 / 212 / 150 | 796 / 477 / 359 (8) | 1,759 | Source: @greencandle9753, 2026-10-21 |
| DeepSeek V4.1 Flash | EXL3 experts + FP8 | DeepSeek V4.1 Flash on 4x 8 GB (GNmac) 1.1.0 | 4 | TP4, x16 | 180 W | MiaAI-Lab | 336 / 247 / 180 | 1,764 | Source: @greencandle9753, 2026-10-23 | |
| DeepSeek V4.1 Flash | EXL3 experts + FP8 | DeepSeek V4.1 Flash on 4x 8 GB (GNmac) 1.2.0 | 4 | TP4, x16, P2P | 180 W | MiaAI-Lab | 356 / 258 / 184 | 868 / 544 / 441 (8) | 2,416 | Source: repo github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx by GNmac-lab on GitHub, posted by @greencandle9753, 2026-10-21 |
| DeepSeek V4.1 Flash | EXL3 experts + FP8 | DeepSeek V4.1 Flash on 4x 8 GB (GNmac) 1.2.0 | 4 | PP4 | 180 W | MiaAI-Lab | 118 / 91 / 63 | 341 / 214 / 163 (8) | 2,311 | Source: repo github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx by GNmac-lab on GitHub, posted by @greencandle9753, 2026-10-21 |
| DeepSeek V4.1 Flash | EXL3 experts + FP8 | DeepSeek V4.1 Flash on 4x 8 GB (GNmac) | 4 | TP4, x16, P2P | 1,024 tok | 212 / 244 / 136 | 425 / 522 / 346 (8) | 2,000 to 2,600 | Source: @darkmaniac7, 2026-10-28 (lab notes) | |
| DeepSeek V4.1 Flash | EXL3 experts + FP8 | DeepSeek V4.1 Flash on 4x 8 GB (GNmac) (localairecipes listing) | 4 | TP4 + P2P, Gen2 x4 | about 10% below GLM 5.3 Flash | Source: @mountains1605, 2026-10-23 | ||||
| DeepSeek V4.1 Flash | all-3-bit EXL3 experts + FP8 | DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build), DSpark off | 4 | TP4, x16, P2P | 200 W | MiaAI-Lab | 105.4 / 105.0 / 104.8 | 559.7 / 482.4 / 449.2 (8) | 2,514 | Source: repo github.com/alanoo81/deepseek-v41-flash-4x-cmp170hx by alanoo81 on GitHub, posted by @alanoo, 2026-10-27 |
| DeepSeek V4.1 Flash | all-3-bit EXL3 experts + FP8 | DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build), DSpark 5 | 4 | TP4, x16, P2P | 200 W | MiaAI-Lab | 369.8 / 267.3 / 200.1 | Source: repo github.com/alanoo81/deepseek-v41-flash-4x-cmp170hx by alanoo81 on GitHub, posted by @alanoo, 2026-10-27 | ||
| DeepSeek V4.1 Flash | EXL3 2.9 experts + FP8/FP4 | DeepSeek V4.1 Flash on 4x 8 GB (CurtisAccelerate TC-MoE) 2026-10-22 | 4 | TP4, P2P off | 140 W | Curtis protocol | 187.2 / 144.2 / 101.2 (count / code / prose) | 271.6 (8, standardized suite) | 1,523 (2K), 1,330 (131K) | Source: repo github.com/CurtisAccelerate/DeepSeekv41-Flash-1017-EXL3-CMP170HX by CurtisAccelerate on GitHub, posted by @anon33anon, 2026-10-20 |
| DeepSeek V4.1 Flash | EXL3 3.5 bpw | DeepSeek V4.1 Flash on 4x 8 GB plus RTX PRO 6000 (ExLlamaV3 TP5) | 4 + 1 other | TP5, native backend | 150 W (CMP) | C coding, mean of 9 | 99.9 | 34.4 per stream (2) | 2,116 (26K), 1,645 (1M) | Source: repo github.com/anoane/09-deepseek-v41-flash-1017-exl3-tp5-1m by anoane on GitHub, posted by @anon33anon, 2026-10-20 |
| DeepSeek V4.1 Flash | official MXFP8 / MXFP4 | DeepSeek V4.1 Flash on 6x 8 GB (TP6) | 6 | TP6, x16 | effort max, end to end incl. reasoning | 78.55 code, 76.56 writing | Source: repo github.com/wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe by wgkbfcp on GitHub, posted by @dale063629, 2026-10-19 | |||
| DeepSeek V4.1 Flash | official | DeepSeek V4.1 Flash on 6x 8 GB (Deepinfer-SM80 TP2 x PP3) | 6 | TP2 x PP3 | 1 stream code | 160.8 | 4,037 (16K), 4,507 (64K) | Source: repo github.com/HalfVulpes/Deepinfer-SM80 by HalfVulpes on GitHub, posted by @reala7096, 2026-10-20 | ||
| DeepSeek V4.1 Flash | official FP4 | own setup | 6 | TP2 + PP3, x4 | code, 1 user | 144.4 | 1,970 cold, 1,987 (131K) | Source: @reala7096, 2026-10-22 | ||
| DeepSeek V4.1 Flash | EXL3 3.5 bpw | own setup | 6 | TP2 + PP3, x4 | code, 1 user | 123.4 | 1,695 cold, 1,647 (131K) | Source: @reala7096, 2026-10-22 | ||
| DeepSeek V4.1 Flash | native quant | own setup | 6 | PP6, x4 | about 67 | Source: @deejay9983, 2026-10-22 | ||||
| DeepSeek V4.1 Flash | not stated | own setup | 6 | prose, c=1 | 157 | Source: @con3po7016, 2026-10-21 | ||||
| DeepSeek V4.1 Flash | official | DeepSeek V4.1 Flash on 8x 8 GB (Deepinfer-SM80 TP4 x PP2) | 8 | TP4 x PP2 | C=1 code | 153.35 | 620.22 (12) | about 3,463; 2,759.9 at 649K | Source: repo github.com/HalfVulpes/Deepinfer-SM80 by HalfVulpes on GitHub, posted by @reala7096, 2026-10-20 | |
| DeepSeek V4.1 Flash | official FP8 | Deepinfer-SM80 vLLM | 8 | PP8, no P2P | 256k ctx | 74 then about 80 | 6,000 (zero cache hit) | Source: @segmond, 2026-10-20 | ||
| DeepSeek V4.1 Flash | official FP4 | own setup | 8 | TP2 x PP4, x4 | code, 1 user | 135 to 139 | 2,520 to 2,690 cold; 2,708 to 2,730 (131K) | Source: @reala7096, 2026-10-22 | ||
| DeepSeek V4.1 Flash | EXL3 3.5 bpw | own setup | 8 | TP2 x PP4, x4 | code, 1 user | 117.1 | 2,025 to 2,126 | Source: @reala7096, 2026-10-22 | ||
| DeepSeek V4.1 Flash | official MXFP4 / MXFP8 | DeepSeek V4.1 Flash on 8x 8 GB (Zanooda PP8) | 8 | PP8, mostly x4 | 180 W | 128k ctx | 117 | 560 (8), 806 to 813 (16) | 4,486 (26K), 6,066 (105K) | Source: repo github.com/Zanooda/deepseek-v4.1-flash-cmp170hx by Zanooda on GitHub, posted by @gnasherrr, 2026-10-17 |
| DeepSeek V4.1 Flash | official MXFP4 / MXFP8 | DeepSeek V4.1 Flash on 8x 8 GB (Zanooda PP8) | 8 | PP8, x16, P2P, Engram disk | 180 W | 1,024 tok | 64 / 46 / 32 (code / struct / prose) | 546 / 316 / 285 (8) | 5,478 (17k) | Source: @darkmaniac7, 2026-10-27 (lab notes) |
| DeepSeek V4.1 Flash | Maya GGUF | DeepSeek V4.1 Flash on 2x 8 GB (Project Maya) | 2 | README figure | about 63 | 500 to 550 (68x) | Source: @pcsv., 2026-10-25 | |||
| Qwen3.8 Flash Next | AutoRound 2/3-bit + FP8 table | Qwen3.8 Flash Next on 1x 8 GB (ElectroHobo 3 bpw) | 1 | x16 | 180 W | MiaAI-Lab | 197.8 / 296.4 / 135.5 | 799.5 / 791.3 / 586.4 (8); 1,384 (48) | 4,243 | Source: repo github.com/ElectroHobo/qwen38-flash-next-3bpw-1x-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-24 |
| Qwen3.8 Flash Next | AutoRound 3 bpw + MTP | Qwen3.8 Flash Next on 1x 8 GB (iIIusi0n SSD PLE) | 1 | PLE on SATA SSD | 180 W | 128 tok, temp 0 | 110.99 | 277 (4), 429 (8), 667 (16) | 2,636 (8K) | Source: repo github.com/iIIusi0n/qwen3.8-flash-next-cmp170hx by iIIusi0n on GitHub, posted by @c133, 2026-10-09 |
| Qwen3.8 Flash Next | AutoRound 3 bpw + MTP | Qwen3.8 Flash Next on 1x 8 GB (iIIusi0n SSD PLE), tuned | 1 | 200 W, NDIV 67, +250 | 5,000+ | Source: @dottoreargus, 2026-10-07 | ||||
| Qwen3.8 Flash Next | AutoRound 3 bpw + MTP | Qwen3.8 Flash Next on 1x 8 GB (iIIusi0n SSD PLE) | 1 | MTP 3, 2,048 chunk | 3,500 | Source: @tekkenz, 2026-10-06 | ||||
| Qwen3.8 Flash Next | EXL3 5.05 bpw | ExLlamaV3, one card | 1 | 69 to 82 | about 1,000 | Source: @tekkenz, 2026-10-06 | ||||
| Qwen3.8 Flash Next | 4.27 bpw GGUF | llama.cpp, MTP | 1 | 60 (ceiling) | Source: @tokukarin, 2026-10-27 | |||||
| Qwen3.8 Flash Next | Unsloth QD-4K_XL, FP16 KV | Strata | 1 | 60+ | Source: @acedogblast, 2026-10-06 | |||||
| Qwen3.8 Flash Next | IQ2 | Strata, quad-channel DDR4 | 1 | no x16 mod | about 90 | 1,600 | Source: @xtj7, 2026-10-15 | |||
| Qwen3.8 Flash Next | IQ3_S | Strata (modified build), 10 GB card at 40 GB | 1 | x16 | 110+ (stock build under 100) | Source: @exile_1ck, 2026-10-07 | ||||
| Qwen3.8 Flash Next | UD-Q3_K_XL | single-kernel decode engine vs llama.cpp | 1 | 70.6 vs 53.4 | Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @c133, 2026-10-20 | |||||
| Qwen3.8 Flash Next | W4A16 Merlin + FP8 table | Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) 0.2.0 | 2 | TP2, x16, P2P | 180 W | MiaAI-Lab | 241.2 / 349.7 / 198.4 | 894.5 / 986.8 / 769.7 (8); 1,937 (128) | 4,696 | Source: repo github.com/ElectroHobo/qwen38-flash-next-w4a16-2x-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-24 |
| Qwen3.8 Flash Next | W4A16 Merlin | Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) 0.1.0 | 2 | TP2 | 253 / 349 / 203 | 842 / 992 / 727 (8) | Source: @nathang6866, 2026-10-20 | |||
| Qwen3.8 Flash Next | W4A16 Merlin | Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) update | 2 | TP2 | max concurrency | 1,814 (128) | Source: @nathang6866, 2026-10-24 | |||
| Qwen3.8 Flash Next | W4A16 Merlin | Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) 0.2.0 | 2 | TP2, x16, P2P | 1,024 tok | 263 / 266 / 215 (code / struct / prose) | 916 / 961 / 729 (8) | 4,600 (17k) | Source: @darkmaniac7, 2026-10-28 (lab notes) | |
| Qwen3.8 Flash Next | W4A16 Merlin | four instances of Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) | 8 | 4x TP2 | 1,024 tok | 3,617 / 3,316 / 2,664 (32, code / struct / prose) | 17,400 total | Source: @darkmaniac7, 2026-10-28 (lab notes) | ||
| Qwen3.8 Flash Next | EXL3 5.05 bpw | Qwen3.8 Flash Next on 2x 8 GB (Morrowmake EXL3) | 2 | TP2 | MiaAI-Lab | 152.8 / 152.6 / 118.3 | 588.2 / 674.7 / 489.9 (8) | 2,651 | Source: repo github.com/ElectroHobo/qwen38-flash-next-w4a16-2x-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-24 | |
| Qwen3.8 Flash Next | EXL3 5.05 bpw | Qwen3.8 Flash Next on 2x 8 GB (Morrowmake EXL3) on a Broadwell host | 2 | TP2 | 55 to 70 | Source: repo github.com/ElectroHobo/qwen38-flash-next-w4a16-2x-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-24 | ||||
| Qwen3.8 Flash Next | AWQ g32 uncensored, INT4 table on SSD | Qwen3.8 Flash Next on 2x 8 GB (mannydelrio PP2) | 2 | PP2, x4, no P2P | 180 W | MiaAI-Lab | 192.4 / 177.8 / 145.4 | 986.4 / 734.2 / 619.6 (8) | 7,006 | Source: repo github.com/mannydelrio/qwen38-flash-next-2x-cmp170hx by mannydelrio on GitHub, posted by @mrmannyatx, 2026-10-06 |
| Qwen3.8 Flash Next | Intel W4A16, table on NVMe | AlexBrandes/cmp170hx-qwen38-flash-next | 2 | PP2, x4 | MiaAI-Lab | 250.8 / 226.2 / 157.4 | 999.5 / 840.4 / 566.8 (8) | 6,300 to 7,100 | Source: repo github.com/ElectroHobo/qwen38-flash-next-w4a16-2x-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-24 | |
| Qwen3.8 Flash Next | W4A16 INT4-PLE | Qwen3.8 Flash Next on 2x 8 GB (DUFRelic PP2) | 2 | PP2, MTP 3, 74 SMs | 140 W | llama-benchy pp16384 tg2048 | 133.39 | 245.17 (4) | 6,213.66 (C1), 6,639.25 (C4) | Source: repo github.com/DUFRelic/qwen38-flash-next-recipe by DUFRelic on GitHub, posted by @dufrelic, 2026-10-05 |
| Qwen3.8 Flash Next | W4A16 | own setup, PP | 2 | PP2 | about 120 | about double (4) | Source: @dufrelic, 2026-10-07 | |||
| Qwen3.8 Flash Next | INT4-mixed AutoRound | Plean's setup | 2 | PP2, MTP | about 100 each (2), about 80 each (3 to 4) | Source: @plean., 2026-10-07 | ||||
| Qwen3.8 Flash Next | AWQ INT4 | Qwen3.8 Flash Next on 2x 8 GB (vektorprime PP2) | 2 | PP2, MTP k=1 | 75.4 to 75.8 (58.1 to 58.6 without MTP) | Source: repo github.com/vektorprime/qwen38-flash-next-pp2 by vektorprime on GitHub, posted by @vektor_prime, 2026-10-07 | ||||
| Qwen3.8 Flash Next | W4A16 AutoRound, BF16 table on NVMe | Qwen3.8 Flash Next on 3x 8 GB (clee PP3) | 3 | PP3, x4, no P2P | cold, temp 0 | 74.4 (1K), 72.9 (1M) | 4,831 (64K), 4,748 (1M) | Source: repo github.com/clee/qwen3.8-flash-next-3x170hx by clee on GitHub, posted by @c133, 2026-10-04 | ||
| Qwen3.8 Flash Next | uncensored variant | Qwen3.8 Flash Next on 3x 8 GB (clee PP3) | 3 / 2 | PP, MTP 1 | agentic | about 90 (3 cards), 100 to 110 (2 cards) | Source: @68x0, 2026-10-09 | |||
| Qwen3.8 Flash Next | own setup | 3 | 110+ | Source: @trustinter, 2026-10-05 | ||||||
| Qwen3.8 27B | W4A16 | Qwen3.8 27B on 1x 8 GB (W4A16 vLLM) | 1 | 262k ctx | about 170 (60 to 70 near long ctx) | Source: @kellertreppe123, 2026-10-05 | ||||
| Qwen3.8 27B | own setup | 1 | about 100 | about 3,700 | Source: @chonsworth, 2026-10-05 | |||||
| Qwen3.8 27B | W4A16 + drafter | 1 | c=1, about 30k ctx | about 200 | Source: @wakeupmbrc1, 2026-10-14 | |||||
| Qwen3.8 27B | DFlash | 1 | code, 16k / 200k ctx | 210+ / 110+ | Source: @wakeupmbrc1, 2026-10-04 | |||||
| Qwen3.8 27B | FP8 + DSpark | sglang | 1 | HLE / MMLU / GSM8K | 200 to 400, avg about 300 (4) | Source: @reggie7756, 2026-10-04 | ||||
| Qwen3.8 27B | FP8 | sglang | 1 | single stream | 60 to 80, spikes 125 | Source: @c133, 2026-10-16 | ||||
| Qwen3.8 27B | INT4 Swift | 1 | 131k ctx, c=3 | about 55 | 200+ (7 sub-agents) | Source: @globalapathy, 2026-10-05 | ||||
| Qwen3.8 27B | W8A8 INT8 | vLLM + MTP | 1 | c=2 | 30 to 40 | Source: @amishrakefightr, 2026-10-05 | ||||
| Qwen3.8 27B | INT kernels | vLLM, DFlash, temp 0 | 1 | 225 | Source: @feralzi, 2026-10-14 | |||||
| Qwen3.8 27B | W4A16 AutoRound + DFlash2 k=7 | vLLM | 1 | 180 W | 136.38 to 140.3 | Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @c133, 2026-10-20 | ||||
| Qwen3.8 27B | Q8 | llama.cpp | 1 | no x16 mod | about 800 | Source: @programmer_007, 2026-10-15 | ||||
| Qwen3.8 27B | 4-bit | own TP4 code (not released) | 4 | TP4 | 190 W | with speculation | 440 | about 4,000 (2,200 without his all-reduce compression) | Source: @turbine, 2026-10-06 | |
| MiMo V2.6 Flash | NVFP4 | MiMo V2.6 Flash on 4x 8 GB | 4 | TP4, x16, P2P | 180 W | MiaAI-Lab | 322.0 / 274.9 / 170.1 | 753.8 / 738.0 / 555.5 (8) | 2,722 | Source: repo github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-18 |
| MiMo V2.6 Flash | NVFP4 | MiMo V2.6 Flash on 4x 8 GB, P2P off | 4 | TP4 | 180 W | MiaAI-Lab | 181.3 / 164.4 / 96.8 | 426.1 / 388.9 / 254.3 (8) | Source: repo github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-18 | |
| Step 5 | own setup | 6 | prose | almost 100 | Source: @con3po7016, 2026-10-17 | |||||
| Qwen 3.6 (comparison, not CMP) | 5-bit | llama.cpp on 2x V100 | 2 V100 | 88 | Source: @xanthefin, 2026-10-18 | |||||
| GLM 5.3 Flash | W4A16 (canada-quant) | GLM 5.3 Flash on 4x 8 GB 1.8.0, ESXi VM | 4 | TP4, Gen2 x8, no P2P, 74 SMs | 180 W | MiaAI-Lab | 361 / 284 / 199 | 697 / 587 / 529 (8) | 1,633-1,753 (32k) | Source: @darkmaniac7, 2026-10-19 (lab notes) |
| GLM 5.3 Flash | W4A16 (canada-quant) | GLM 5.3 Flash on 4x 8 GB 1.8.0, ESXi VM | 4 | PP4, Gen2 x8, no P2P, 74 SMs | 180 W | MiaAI-Lab | 201 / 149 / 97 | 664 / 548 / 477 (8) | 6,834-6,977 (32k) | Source: @darkmaniac7, 2026-10-19 (lab notes) |
| GLM 5.3 Flash | W4A16 (canada-quant) | GLM 5.3 Flash on 4x 8 GB 1.8.0, no drafter | 4 | TP4 / PP4, Gen2 x8 | 180 W | MiaAI-Lab | 101 (TP4), 48 (PP4) | 470 / 419 / 422 (8, TP4) | Source: @darkmaniac7, 2026-10-19 (lab notes) | |
| GLM 5.3 Flash | W4A16 abliterated (own requant) | GLM 5.3 Flash on 4x 8 GB 1.8.0 engine | 4 | TP4, Gen2 x8 | 180 W | MiaAI-Lab | 349 / 254 / 181 | Source: @darkmaniac7, 2026-10-19 (lab notes) | ||
| GLM 5.3 Flash | W4A16 abliterated | Morrowmake engine, 2 CMP + 2 RTX PRO 6000 | 2 + 2 other | PP2 x TP2, CMPs stage 0 | 180 W (CMP) | MiaAI-Lab | 349 / 261 / 180 | 895 / 703 / 611 (8) | 5,495 (32k) | Source: @darkmaniac7, 2026-10-19 (lab notes) |
| GLM 5.3 Flash | W4A16 | Morrowmake engine, 4 CMP + 2 RTX PRO 6000 | 4 + 2 other | PP3 x TP2 | 180 W (CMP) | code / JSON / prose | 224-235 code, 101-110 prose | 553 / 484 / 425 (8) | Source: @darkmaniac7, 2026-10-19 (lab notes) | |
| GLM 5.3 Flash | AWQ W4A16 | PixelML club recipe of record (older stack) | 4 | PP4 (14,12,12,7), MTP k=3 | c=1 | 87.6 | Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @c133, 2026-10-20 | |||
| DeepSeek V4.1 Flash | official MXFP4 / MXFP8 | DeepSeek V4.1 Flash on 8x 8 GB (Zanooda PP8), Engram on NVMe | 8 | PP8 | 105k prompt | 4,455-4,930 | Source: @gnasherrr, 2026-10-17 | |||
| GLM 5.3 (full) | own setup, after a week of tuning | not stated (reply in an 8-card thread) | 60 code / reasoning, 50 prose | Source: @impactic, 2026-10-12 | ||||||
| DeepSeek V4.1 Flash (1017) | not stated | own setup | 6 (posted as "6x single streams": six cards or six streams, not clear) | single stream | 80 to 100 | about 5,000 | Source: @ob88594, 2026-10-24 | |||
| Qwen3.8 Flash Next | not stated | own setup (not stated) | not stated | c=1 | 75 | 3,000+ | Source: @c133, 2026-10-24 | |||
| MiniMax M3.1 | not stated | own setup | 4 (GPU types not stated) | c=1 | 76 | Source: @con3po7016, 2026-10-25 | ||||
| MiniMax M3.1 | not stated | own setup | 6 (GPU types not stated) | c=1 | 30 | Source: @segmond, 2026-10-25 | ||||
| Nanbeige4.3-33B | Q6_K GGUF (dealignai uncensored) | not stated | 1 (10 GB card at 40 GB) | about 200K context, still tuning | 27 (40 posted earlier as "optimized") | Source: @feralzi, 2026-10-25 Source: @feralzi, 2026-10-26 Source: @feralzi, 2026-10-26 |
Compute and memory peaks
Reported 10 GB @djpandaman (mrpandagod) ran clpeak --cuda on an unlocked 10 GB card
that reports sm_80, 122 SMs and 40,483 MB of VRAM (CUDA driver 13.3, toolkit 12.0). He believes the card ran at stock
power settings but had not double-checked. He finds clpeak a better view of what the card can do than OpenCL-benchmark.
| Test | Result |
|---|---|
| FP32 (vector) | 21.9 TFLOPS |
| FP16 (vector, half / half2) | 43.7 / 43.8 TFLOPS |
| FP64 (vector) | 11.0 TFLOPS |
| FP16 x FP16 + FP32 and BF16 x BF16 + FP32 (vector) | 21.9 TFLOPS each |
| INT32 (IMAD) | 21.9 TOPS |
| INT8 dot product (dp4a) | 87.3 to 87.5 TOPS |
| Tensor cores (WMMA / mma.sync) | FP16 312 TFLOPS (331 with FP16 accumulate), BF16 312, TF32 156, FP64 21.4 TFLOPS; INT8 605 TOPS (607 at k32), INT4 647 TOPS |
| cuBLASLt GEMM peak | FP16 302, BF16 308, TF32 146, FP32 21.6, FP64 21.9 TFLOPS; INT8 424 TOPS |
| Global memory bandwidth (float / float2 / float4) | 1.68 / 1.74 / 1.82 TB/s |
| Local memory bandwidth (float / float2 / float4) | 10.7 / 17.1 / 18.3 TB/s |
| Image memory bandwidth (float4) | 1.64 TB/s |
| Pinned host transfers | 12.3 GB/s host to device, 13.2 GB/s device to host |
| Kernel launch round trip | 6.79 us |
Source: @djpandaman, 2026-10-23
Source: @djpandaman, 2026-10-23
More on this card variant: 10 GB card and Full SM unlock.
Not in the table
- Image and video generation (minutes per clip): see Recipes#Other workloads.
- Mining hash rates (TH/s): see Cooling and power and Tools.
- Kernel microbenchmarks (Deepinfer-SM80, ExLlamaV3 GEMM): see Engines.
- Long write-ups with numbers that wait for their authors (Plean, matstatman, Feralzi): see Recipes.