Recipes
From CMP Wiki
More actions
A recipe is a model running on CMP 170HX cards with the exact settings and the numbers that came out. Each recipe page has an infobox (model, quant, engine, cards, topology, tok/s, power, status), the steps, the measured numbers with their conditions, known issues and credits. All recipes lists every recipe page as a sortable table.
How to read the numbers: unless a page says otherwise, the infobox decode is one user on code, and prefill is
a cold prompt read. Most recipe authors use the MiaAI-Lab protocol (400-token answers, temperature 0, thinking off) and
report three columns, structured / code / prose; code is usually the middle one and prose the slowest. Numbers from
different protocols, power caps (140 to 250 W) and link widths (x4 or x16) are not directly comparable. Every number with
its setup is on Benchmarks.
Almost everything here runs on the 8 GB card unlocked to 64 GB with the 74-SM unlock (see 8 GB card and Driver unlock). No recipe has been published for the 10 GB card yet; see the variant section below.
Quick picks by card count
| Cards | Model | Recipe | One user (code) | Notes |
|---|---|---|---|---|
| 1 | Qwen3.8 Flash Next | Qwen3.8 Flash Next on 1x 8 GB (ElectroHobo 3 bpw) | 296 tok/s | needs 96 GB host RAM (or HMM disk mode) and an x16 card; the community's go-to for one card |
| 1 | Qwen3.8 Flash Next | Qwen3.8 Flash Next on 1x 8 GB (iIIusi0n SSD PLE) | 111 tok/s | runs with 15 GiB host RAM; PLE table on SSD; confirmed |
| 1 | Qwen3.8 27B (dense) | Qwen3.8 27B on 1x 8 GB (W4A16 vLLM) | about 170 tok/s | reported, one run (a second run only partly matches) |
| 2 | Qwen3.8 Flash Next | Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) | 350 tok/s | x16 + P2P; up to 128 users; confirmed by Ivan |
| 2 | Qwen3.8 Flash Next | Qwen3.8 Flash Next on 2x 8 GB (mannydelrio PP2) | 178 tok/s | stock x4 cards, no P2P, 16 GB host RAM; fastest prompt reading (7,006 tok/s) |
| 2 | GLM 5.3 Flash | GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 TP2) | 108 tok/s | everything in HBM, 393K context, one or two requests at a time |
| 3 | Qwen3.8 Flash Next | Qwen3.8 Flash Next on 3x 8 GB (clee PP3) | 74 tok/s | unmodded x4 cards, 1M context, 7.36M KV; confirmed |
| 3 | GLM 5.3 Flash | GLM 5.3 Flash on 3x 8 GB | not posted | Morrowmake's recipe at PP3 with an 8-bit KV cache |
| 4 | GLM 5.3 Flash | GLM 5.3 Flash on 4x 8 GB | 442 tok/s | the community daily driver; confirmed |
| 4 | DeepSeek V4.1 Flash | DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake) | up to 287 tok/s | about 240 GiB host RAM and 680 GiB disk; confirmed |
| 4 | DeepSeek V4.1 Flash | DeepSeek V4.1 Flash on 4x 8 GB (GNmac) | 258 tok/s | less RAM (Engram on NVMe); confirmed; see its long-context copy issue |
| 4 | MiMo V2.6 Flash | MiMo V2.6 Flash on 4x 8 GB | 275 tok/s | 1M context, image input |
| 6 | DeepSeek V4.1 Flash (full checkpoint) | DeepSeek V4.1 Flash on 6x 8 GB (Deepinfer-SM80 TP2 x PP3) | 161 tok/s | native precision, 4.6M KV |
| 8 | DeepSeek V4.1 Flash (full checkpoint) | DeepSeek V4.1 Flash on 8x 8 GB (Zanooda PP8) | 117 tok/s | best prefill (about 6k tok/s), 6.2M KV, 1M context |
| 8 | GLM 5.3 (full) | GLM 5.3 on 8x 8 GB (vllm-170hx-glm5) | 49 tok/s | 8 to 10 cards, 1M context; confirmed |
| 8 | GLM 5.3 Flash, two instances | GLM 5.3 Flash on 4x 8 GB#Eight cards | about 1,600 tok/s aggregate |
"Confirmed" means the page has a confirmation by a second person at import time; the badge on each page is authoritative.
All recipe pages
| Model | Cards | Engine | Page |
|---|---|---|---|
| GLM 5.3 Flash | 4 | Morrowmake vLLM fork, W4A16 | GLM 5.3 Flash on 4x 8 GB |
| GLM 5.3 Flash | 3 | Morrowmake vLLM fork, W4A16 PP3 | GLM 5.3 Flash on 3x 8 GB |
| GLM 5.3 Flash | 2 | ExLlamaV3 TP2, EXL3 3.05 bpw | GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 TP2) |
| GLM 5.3 Flash | 2 | ExLlamaV3 layer split, EXL3 3.05 bpw | GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split) |
| GLM 5.3 Flash | 2 | Project Maya, GGUF | GLM 5.3 Flash on 2x 8 GB (Project Maya) |
| GLM 5.3 (full) | 8 to 10 | Morrowmake ampere-glm53 + patches, PP8 | GLM 5.3 on 8x 8 GB (vllm-170hx-glm5) |
| GLM 5.3 (full) | 8 | vLLM PP8, Int4-Int8Mix | GLM 5.3 on 8x 8 GB (Int4-Int8Mix PP8) |
| DeepSeek V4.1 Flash | 4 | Morrowmake vLLM fork, EXL3 experts | DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake) |
| DeepSeek V4.1 Flash heretic | 4 | Morrowmake vLLM fork + in-place abliteration patch | DeepSeek V4.1 Flash heretic on 4x 8 GB |
| DeepSeek V4.1 Flash | 4 | vLLM + sm_80 patch stack (GNmac) | DeepSeek V4.1 Flash on 4x 8 GB (GNmac) |
| DeepSeek V4.1 Flash | 4 | GNmac + fidelity patches | DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build) |
| DeepSeek V4.1 Flash | 4 | Morrowmake engine + TC-MoE kernels | DeepSeek V4.1 Flash on 4x 8 GB (CurtisAccelerate TC-MoE) |
| DeepSeek V4.1 Flash | 4 + RTX PRO 6000 | ExLlamaV3 + TabbyAPI TP5 | DeepSeek V4.1 Flash on 4x 8 GB plus RTX PRO 6000 (ExLlamaV3 TP5) |
| DeepSeek V4.1 Flash | 6 | custom SM80 vLLM, TP6 | DeepSeek V4.1 Flash on 6x 8 GB (TP6) |
| DeepSeek V4.1 Flash | 6 | Deepinfer-SM80, TP2 x PP3 | DeepSeek V4.1 Flash on 6x 8 GB (Deepinfer-SM80 TP2 x PP3) |
| DeepSeek V4.1 Flash | 8 | Deepinfer-SM80, TP4 x PP2 | DeepSeek V4.1 Flash on 8x 8 GB (Deepinfer-SM80 TP4 x PP2) |
| DeepSeek V4.1 Flash | 8 | vLLM PR #56201 + 20 patches, PP8 | DeepSeek V4.1 Flash on 8x 8 GB (Zanooda PP8) |
| DeepSeek V4.1 Flash | 2 | Project Maya | DeepSeek V4.1 Flash on 2x 8 GB (Project Maya) |
| Qwen3.8 Flash Next | 1 | Morrowmake image + patches, 3 bpw | Qwen3.8 Flash Next on 1x 8 GB (ElectroHobo 3 bpw) |
| Qwen3.8 Flash Next | 1 | patched vLLM, PLE on SSD | Qwen3.8 Flash Next on 1x 8 GB (iIIusi0n SSD PLE) |
| Qwen3.8 Flash Next | 2 | Morrowmake image + patches, W4A16 TP2 | Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) |
| Qwen3.8 Flash Next | 2 | Morrowmake vLLM fork, EXL3 | Qwen3.8 Flash Next on 2x 8 GB (Morrowmake EXL3) |
| Qwen3.8 Flash Next | 2 | wtdcode/vllm-backport PP2 | Qwen3.8 Flash Next on 2x 8 GB (mannydelrio PP2) |
| Qwen3.8 Flash Next | 2 | vLLM v0.30.0 + INT4-PLE plugin PP2 | Qwen3.8 Flash Next on 2x 8 GB (DUFRelic PP2) |
| Qwen3.8 Flash Next | 2 | vLLM PR #53899 + patches PP2 | Qwen3.8 Flash Next on 2x 8 GB (vektorprime PP2) |
| Qwen3.8 Flash Next | 3 (or 2) | vLLM nightly + clee/vllm-sm80 PP3 | Qwen3.8 Flash Next on 3x 8 GB (clee PP3) |
| Qwen3.8 27B | 1 | vLLM W4A16 | Qwen3.8 27B on 1x 8 GB (W4A16 vLLM) |
| MiMo V2.6 Flash | 4 | Morrowmake image, NVFP4 TP4 | MiMo V2.6 Flash on 4x 8 GB |
Longer write-ups that are not reproduced here until their authors agree:
- Plean's Qwen3.8 Flash Next on 2x CMP 170HX with vLLM PP, MTP and BF16 PLE offload. A longer write-up by @plean. exists; see Discord.
- matstatman's Qwen3.8 Flash Next AWQ W4A16 on 4x CMP 170HX with wtdcode/vllm-backport and 32 GB host RAM. A longer write-up by @matstatman exists; see Discord.
- Feralzi's Qwen 3.8 27B single-card overclock setup. A longer write-up by @feralzi exists; see Discord.
Other published recipes not yet written up as pages: the localairecipes entry "DeepSeek-V4.1-Flash (764B) on 4x CMP 170HX,
PP4, shadow KV, EXL3 2bpw" (Miséricorde's critique: the author has no P2P, uses FP8 KV at about 4 KB per token where the
native FP4 KV is about 0.9 KB, and 2 bpw is unnecessary because 3.05 to 3.15 bpw fits), PixelML/club-170hx's DeepSeek
V4.1 Flash EXL3 4-card TP4 EP4 DSpark result, AlexBrandes/cmp170hx-qwen38-flash-next (two cards, PP2, x4 links) and
kaka86mm/vllm-mimo-170hx.
Source: @wakeupmbrc1, 2026-10-17
Source: @c133, 2026-10-20
Choosing a model
The community view as of late October 2026 (a rough consensus, with dissent):
- One card: Qwen3.8 Flash Next with the PLE patches is the go-to; the dense Qwen3.8 27B is the alternative. Source: @68x0, 2026-10-27
- Two cards: Qwen3.8 Flash Next. Miséricorde: "3.8fn for 2 is the best", and for two cards it beats any GLM recipe because GLM needs four cards to run properly in integer weights. Source: @wakeupmbrc1, 2026-10-13
Source: @wakeupmbrc1, 2026-10-26 - Four cards: GLM 5.3 Flash W4A16 on Morrowmake's recipe, with DeepSeek V4.1 Flash 1017 as an option if you have the RAM or SSD for its Engram tables. Four cards give disproportionately more than two because the whole "Flash" model stack fits; Miséricorde found even his best overclocked Qwen Flash Next 1.6x slower than GLM Flash on stock 180 W cards. Source: @wakeupmbrc1, 2026-10-16
Source: @wakeupmbrc1, 2026-10-18
Source: @lia_oc, 2026-10-26 - Six cards: DeepSeek V4.1 Flash at native precision (TP2 x PP3). 384 GB should be enough for DeepSeek V4.1 Flash and more than enough for GLM 5.3 Flash. Theory Miséricorde thinks six may be worse than four for DeepSeek because TP6 is impossible. Source: @wakeupmbrc1, 2026-10-17
Source: @civiloid, 2026-10-19 - Eight cards: full GLM 5.3, DeepSeek V4.1 Flash at native precision with the best prefill, or two 4-card instances of GLM 5.3 Flash for throughput. Two Flash instances beat one full GLM 5.3: much faster and not really different in capability (Miséricorde). Source: @wakeupmbrc1, 2026-10-16
- Five cards are mostly wasted: vLLM's TP must divide the model's attention head count (DeepSeek V4.1: TP 1, 2, 4 or 8), so buy 4 or 6. A fifth card can still hold mmproj or a draft model in llama.cpp. Source: @ob88594, 2026-10-21
Source: @greencandle9753, 2026-10-21
Source: @da6b, 2026-10-21
Contested Model quality, by person:
- GLM 5.3 Flash against Qwen Flash Next on four cards: Mountains finds GLM a step up in logic; OB88594 sums up Qwen for world knowledge, GLM for coding; segmond thinks Qwen 3.8 Flash is excellent and GLM will not beat it everywhere. Source: @mountains1605, 2026-10-04
Source: @segmond, 2026-10-04
Source: @ob88594, 2026-10-06 - DeepSeek V4.1 Flash against GLM 5.3 Flash: Mountains found DeepSeek a bit more intelligent but less consistent at first, then rated Morrowmake's DeepSeek build above GLM Flash; DeeJay's own suite had DeepSeek 1017 ahead of GLM 5.3 Flash; reala thinks full GLM 5.3 may still win pure coding. Source: @deejay9983, 2026-10-22
Source: @mountains1605, 2026-10-23
Source: @mountains1605, 2026-10-27 - Qwen Flash Next against the 27B: PJ finds the 27B better at tool calling and reasoning; dash got a better game out of Flash Next IQ3_S than 27B Q6; Programmer007: Next builds, 27B fixes bugs. orrorin: low-quant Flash Next and high-quant 27B are neck and neck in benchmarks; dottoreargus: Flash Next is much faster for the same knowledge. Source: @_orrorin, 2026-10-06
Source: @dottoreargus, 2026-10-06
Source: @zingy90210, 2026-10-17
Source: @mowgli8628, 2026-10-17
Source: @programmer_007, 2026-10-17 - MiMo V2.6 Flash: see MiMo V2.6 Flash on 4x 8 GB#Is it better than GLM 5.3 Flash?.
- Context in practice: matstatman runs Qwen3.8 Flash on four cards at the normal 265k context and finds compaction fast enough that he does not need 1M. Source: @matstatman, 2026-10-20
- Step 5: Con3po runs it at almost 100 tok/s prose with 1M context on six cards and keeps it for vision. Source: @con3po7016, 2026-10-17
Choosing a layout and engine
- Use vLLM (or sglang), not llama.cpp. llama.cpp is too slow on these cards; the CMP prefers INT4/INT8 weights (W4A16 through Marlin), which vLLM serves well. See Engines and Quantization on the 170HX. Source: @ob88594, 2026-10-06
Source: @gnasherrr, 2026-10-26 - TP needs x16 links (and ideally P2P); PP tolerates x4. On PCIe Gen2 x4, TP4 gives usable decode but unusable prefill; PP passes only activations. With x16 and P2P, TP4 wins decode for GLM Flash and MiMo, and P2P off costs about 40% on MiMo TP4. See Unlocks for the x16 mod and P2P. Source: @voidgiraffe, 2026-10-04
Source: @wakeupmbrc1, 2026-10-18 - Contested Jeremy (Ends): a dense model in TP over PCIe will be bandwidth-choked, so PP with P2P DMA is the safer default; Miséricorde: TP4 viability depends on the model's architecture (GLM Flash is latency-bound and fastest in TP4). Plean: TP is repeatedly shown to be PCIe-limited on these cards. NikhilR finds TP2 x PP2 better than TP4 on four cards for A3B models. Source: @plean., 2026-10-06
Source: @nikhilr1696, 2026-10-07
Source: @endsjeremy, 2026-10-18
Source: @wakeupmbrc1, 2026-10-18 - Host RAM and NVMe matter for the n-gram models: Qwen3.8 Flash Next's PLE table is about 100 GB in BF16 (48 GiB in FP8) and DeepSeek V4.1's Engram tables about 189 GiB. In RAM a lookup costs about 0.3 ms per token, from SSD about 1 to 1.5 ms; RAM speed itself is irrelevant because the lookups are latency-bound. See Quantization on the 170HX. Source: @wakeupmbrc1, 2026-10-11
Source: @wakeupmbrc1, 2026-10-17 - Power: most published numbers are at 180 W per card. See Cooling and power.
- Mixing 8 GB and 10 GB cards: you are limited by the slowest card. Theory Source: @moliminous, 2026-10-12
- Theory Takka's idea for prefill on Gen2 cards: keep a TP copy in VRAM and a PP copy in host RAM and swap the whole model to PP for large prefills (10 to 15 s per swap on Gen2); untested. Source: @spectakkular, 2026-10-04
10 GB card
No recipe in this list was measured on the 10 GB card. What exists:
- eXile runs Qwen3.8 Flash Next (IQ3_S) with Strata on a single 10 GB-variant card configured at 40 GB (14600K, 128 GB DDR4); moving it from x4 to x16 mainly improved prefill. See Benchmarks and 10 GB card. Source: @exile_1ck, 2026-10-07
- Theory With an 80 GB unlock, four cards (320 GB) would hold the full GLM 5.3 EXL3 quant, which is under 300 GB. Source: @stcord, 2026-10-09
Source: @globalapathy, 2026-10-09
If you run a recipe on 10 GB cards, add a confirmation on its page with the card variant set to 10 GB.
Other workloads
- Video and image generation: MiniMax H3 makes an 8-second clip in 10 to 15 minutes on one unlocked 8 GB card (domrockt2); Phoenix17 makes 30-second clips at 640x960 in about 15 minutes with a custom H3 model and says much longer will not fit on one card; LTX 2.5 is lighter with a better latent upscaler. quark2world made a full music video with H3; Qwen Image 2.1 needs about 40 GB. Takka recommends Breeze for one-shot voice cloning. Source: @phoenixone7, 2026-10-04
Source: @quark2world, 2026-10-04
Source: @spectakkular, 2026-10-06
Source: @domrockt2, 2026-10-08 - Training: a single CMP trained almost 4x slower than a 5090 (same nominal BF16 TFLOPs); programmerboi suspects no tuned GEMM kernel exists for the harvested die. orrorin's SFT on four cards was slow and PCIe 2.0 x4 was the bottleneck during offload, so the x16 mod is essential; prototype locally and rent for serious training. Source: @programmerboi, 2026-10-16
Source: @_orrorin, 2026-10-16 - Mining between jobs: Pearl mining runs at 145 to 164 TH/s per card at about 200 W; lower HBM NDIV with REFRESH 24 saves heat without hash loss but must be raised again for LLM work. RH-3D's water-cooled 170tune settings:
--ndiv 66 --offset 270 --clk 1410 --timings "REFRESH 24"at about 202 W (an earlier--ndiv 66 --offset 240 --clk 1380gave 164 TH/s at about 200 W). Source: @rh3dai, 2026-10-08
Source: @rh3dai, 2026-10-05 See Cooling and power and Tools. Source: @analyticeth, 2026-10-04
Source: @markshowa, 2026-10-26
Not yet run on these cards
- Theory unsloth/flint-GGUF (39.8B, 2.5B active): Programmer007 expects Q8 to fit one card and be very fast. Source: @programmer_007, 2026-10-22
- Theory MiniMax 3.1 Flash at 4 bits should not fit four cards (quants keep parts at BF16 or FP8, so expect more than 250 GB) unless it uses the same n-gram approach as DeepSeek V4.1 Flash. Source: @civiloid, 2026-10-21
- Theory Open question from cibernox for a company build: is 8x CMP 170HX still the most convenient way to 512 GB of VRAM after the October price rises (16 V100s being too inconvenient)? No answer yet. Source: @cibernox, 2026-10-19
- Theory roboalchemist is packing an FP4 DeepSeek V4.1 1017 mix sized for six cards. Source: @roboalchemist., 2026-10-17