Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

All recipes: Difference between revisions

From CMP Wiki
More actions
← Older editNewer edit →
VisualWikitext
Wiki lane: phone-width tables and infobox, header v2
Wiki lane: readable list tables (fewer columns, one-line cells), Cards per host on platforms
Line 1: Line 1:
{{#cmpcache:300}}Every recipe on the wiki: a model running on CMP 170HX cards, with the settings and the numbers. Click a column to
{{#cmpcache:300}}Every recipe on the wiki: a model running on CMP 170HX cards, with the settings and the numbers. Click a column
sort; type in the search box to filter (for example <code>10 GB</code> or <code>vLLM</code>).
heading to sort; long values are cut to one line, hover for the full text or open the recipe.


'''Filter by card variant:''' [{{fullurl:Special:Drilldown/Recipe|cards={{urlencode:8 GB}}}} 8 GB only] ·
'''Filter by card variant:''' [{{fullurl:Special:Drilldown/Recipe|cards={{urlencode:8 GB}}}} 8 GB only] ·
Line 10: Line 10:
{{#forminput:form=Recipe|size=50|placeholder=Page name, e.g. Qwen3-32B AWQ on vLLM, 4x 10 GB|button text=Add a recipe|autocomplete on category=Recipes}}
{{#forminput:form=Recipe|size=50|placeholder=Page name, e.g. Qwen3-32B AWQ on vLLM, 4x 10 GB|button text=Add a recipe|autocomplete on category=Recipes}}


<div class="cmpw-phone-only">On a phone this table shows the recipe, card variant, decode speed and status. Open a recipe for the rest, or turn the phone sideways.</div>
<div class="cmpw-phone-only">On a phone this table shows the recipe, cards, decode speed and status. Open a recipe for the rest, or turn the phone sideways.</div>
<div class="cmpw-list cmpw-list--recipes">
<div class="cmpw-list cmpw-list--recipes">
{{#cargo_query:tables=Recipe
{{#cargo_query:tables=Recipe
|fields=_pageName=Recipe,model=Model,quant=Quant,engine=Engine,cards=Variant,card_count=Cards,topology=Topology,context=Context,decode_tps=Decode tok/s,prefill_tps=Prefill tok/s,vram_per_card=VRAM per card,power_cap=Power cap,driver=Driver,status=Status,tested_by=Tested by,tested_on=Tested on
|fields=_pageName=page,model,quant,engine,cards,card_count,context,decode_tps,prefill_tps,power_cap,status,tested_by,tested_on
|order by=Recipe.decode_tps DESC
|order by=Recipe.decode_tps DESC
|limit=1000
|limit=1000
|format=dynamic table
|format=template
|rows per page=50
|template=Recipe list row
|named args=yes
|intro=<table class="wikitable sortable cmpw-listtable"><tr><th>Recipe</th><th>Model</th><th>Quant</th><th>Engine</th><th>Cards</th><th>Context</th><th>Decode tok/s</th><th>Prefill tok/s</th><th>Power cap</th><th>Status</th><th>Tested by</th><th>Tested on</th></tr>
|outro=</table>
|default=No recipes yet. Add the first one with the box above.
|default=No recipes yet. Add the first one with the box above.
}}</div>
}}</div>

Revision as of 09:40, 30 October 2026

Every recipe on the wiki: a model running on CMP 170HX cards, with the settings and the numbers. Click a column heading to sort; long values are cut to one line, hover for the full text or open the recipe.

Filter by card variant: 8 GB only · 10 GB only · recipes for both · Category:Recipes for 8 GB · Category:Recipes for 10 GB. Status is computed: Proven means someone else reproduced it (see Status taxonomy).


On a phone this table shows the recipe, cards, decode speed and status. Open a recipe for the rest, or turn the phone sideways.
RecipeModelQuantEngineCardsContextDecode tok/sPrefill tok/sPower capStatusTested byTested on
GLM 5.3 Flash on 4x 8 GBGLM-5.3-Flash (320B MoE, 18B active)W4A16 (INT4 weights, group size 128, FP16 activations): canada-quant/GLM-5.3-Flash-W4A16-MTPMorrowmake/vllm-cmp170hx (vLLM fork, container image)8 GB262,144Proven
Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2)Qwen3.8-Flash-Next (125B MoE per the recipe)W4A16: halt95/Qwen3.8-Flash-Next-W4A16-Merlin (AutoRound INT4 g128 experts, INT8 GDN projections, FP8 n-gram table)Morrowmake/vllm-cmp170hx image (Qwen EXL3 release) with patch files mounted read-only8 GB262,144Proven
Qwen3.8 Flash Next on 1x 8 GB (ElectroHobo 3 bpw)Qwen3.8-Flash-Next (125B MoE per the recipe)AutoRound mixed 2/3-bit (klee100, 2.998 effective bpw, 47 GiB on card) plus halt95's FP8 n-gram table in host RAMMorrowmake/vllm-cmp170hx image with patch files mounted read-only8 GB262,144Reported
DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake)DeepSeek-V4.1-Flash-1017Hybrid: routed experts EXL3 2.9 bpw (LibertAIDAI), everything else official FP8/BF16; KV BF16Morrowmake/vllm-cmp170hx (vLLM fork, container image)8 GB262,144Reported
MiMo V2.6 Flash on 4x 8 GBMiMo-V2.6-Flash-RL (309B MoE, 15B active; text, image, video and audio input)NVFP4 (ModelOpt mixed precision): ProCreations/MiMo-V2.6-Flash-RL-NVFP4, run as 4-bit weights with BF16 activations through MarlinMorrowmake/vllm-cmp170hx image (tag qwen38-recipe-1.0.0) plus one mounted file (vLLM #58235 vision-sink fix)8 GB1,048,576Reported
DeepSeek V4.1 Flash on 4x 8 GB (GNmac)DeepSeek-V4.1-Flash-1017Hybrid: routed experts EXL3 (mostly 3 bpw, 2 bpw on layers 18-22) from LibertAI's 2.9 bpw pack; attention, shared experts, indexer and DSpark drafter official FP8; KV fp8_ds_mlavLLM with a pinned sm_80 patch stack (built locally from source)8 GB262,144 (up to 1,048,576)Proven
Qwen3.8 Flash Next on 2x 8 GB (Plean PP2)Qwen3.8-Flash-NextMixed INT4 AutoRound (Minachist/Qwen3.8-Flash-Next-INT4-Mixed-AutoRound, about 65 GB) with the full BF16 PLE table on NVMevLLM 0.29.1rc1 nightly plus the checkpoint author's patches8 GB262,144 (256K in the post)Reported
Qwen3.8 Flash Next on 2x 8 GB (mannydelrio PP2)Qwen3.8-Flash-Next-Uncensored (refusal-removed, from orcarouter)AWQ W4A16 g32 (leoncca/Qwen3.8-Flash-Next-Uncensored-AWQ-g32); PLE table converted to INT4 and served from SSDwtdcode/vllm-backport 3142ded1 plus a two-file patch and two tinybeast PLE plugins (venv, no container)8 GB262,144Reported
Qwen3.8 27B on 1x 8 GB (W4A16 vLLM)Qwen3.8-27B (dense)W4A16 (INT4 weights, Marlin kernels)vLLM8 GB262,144 native (YaRN to about 500k in the recipe's long variant)Reported
DeepSeek V4.1 Flash on 6x 8 GB (Deepinfer-SM80 TP2 x PP3)DeepSeek-V4.1-Flash-1017Official checkpoint (FP8 dense, FP4 experts); software FP8 on sm_80vLLM PR #56201 head (79a7108d) plus the two Deepinfer-SM80 patches and 30 drop-in modules8 GB1,048,576Reported
DeepSeek V4.1 Flash heretic on 4x 8 GBDeepSeek-V4.1-Flash-1017 heretic (pqhaz abliteration)Hybrid: routed experts EXL3 2.9 bpw (LibertAIDAI), everything else official FP8/BF16 with the 38 heretic attn.wo_b tensorsMorrowmake/vllm-cmp170hx (vLLM fork, container image)8 GB262,144Reported
DeepSeek V4.1 Flash on 8x 8 GB (Deepinfer-SM80 TP4 x PP2)DeepSeek-V4.1-Flash-1017Official checkpoint (FP8 dense, FP4 experts); software FP8 on sm_80vLLM PR #56201 head (79a7108d) plus the two Deepinfer-SM80 patches and 30 drop-in modules8 GB1,048,576Reported
Qwen3.8 Flash Next on 2x 8 GB (Morrowmake EXL3)Qwen3.8-Flash-NextEXL3 5.05 bpw; PLE (n-gram) table read from SSDMorrowmake/vllm-cmp170hx (EXL3 path)8 GB262,144Reported
DeepSeek V4.1 Flash on 4x 8 GB (CurtisAccelerate TC-MoE)DeepSeek-V4.1-Flash-1017Hybrid: LibertAI EXL3 2.9 bpw routed experts (layers 0-39), official FP8/FP4 for attention, shared experts, head, draft and Engram; KV fp8_ds_mlaMorrowmake/vllm-cmp170hx (bind-mounted sm80 package) plus the author's tensor-core MoE kernels8 GB262,144 (weights address 1,048,576)Reported
Qwen3.8 Flash Next on 2x 8 GB (DUFRelic PP2)Qwen3.8-Flash-NextW4A16 INT4 g128 with the PLE table repacked INT4 g32: zankich/Qwen3.8-Flash-Next-W4A16-Merlin-INT4PLE-FP8KV; KV BF16vLLM v0.30.0 + zankich INT4-PLE plugin + six hash-guarded patch layers8 GB262,144Reported
DeepSeek V4.1 Flash on 8x 8 GB (Zanooda PP8)DeepSeek-V4.1-Flash-1017Official checkpoint as downloaded: MXFP4 routed experts as Marlin W4A16, MXFP8 dense as Marlin W8A16; KV fp8vLLM PR #56201 head (79a7108d) plus a 20-patch sm_80 series; image zanooda/vllm-sm80-ds41f:v41-sm808 GB1,048,576Reported
Qwen3.8 Flash Next on 4x 8 GBQwen3.8-Flash-NextAWQ W4A16: wtdcode/Qwen3.8-Flash-Next-AWQ-W4A16wtdcode/vllm-backport (image lazymio/vllm-backport:latest-sm80)8 GB1,000,000 (YaRN x4 from 262,144)Reported
Qwen3.8 Flash Next on 1x 8 GB (iIIusi0n SSD PLE)Qwen3.8-Flash-NextAutoRound 3 bpw + MTP: klee100/Qwen3.8-Flash-Next-AutoRound-3bpw-MTP; 95.37 GiB BF16 PLE table on SSDvLLM a5a30471 plus a 13-file downstream patch (native Linux AIO PLE reader)8 GB262,144 advertised (tested through 32K)Proven
GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 TP2)GLM-5.3-Flash (320B MoE, 18B active)EXL3 3.05 bpw: turboderp/GLM-5.3-Flash-exl3; DFlash2 drafter converted to EXL3 6 bpwExLlamaV3 (patched, pinned)8 GB392,960 per request; 655,360-token Q8 KV pool shared by two requestsReported
DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build)DeepSeek-V4.1-Flash-1017Hybrid: all 40 expert layers at 3 bits (LibertAI 2.9 bpw pack with layers 18-22 grafted from the 3.5 bpw pack), official FP8 dense tensorsGNmac-lab recipe image 1.2.0 (vLLM with sm_80 patches) plus three patches8 GB262,144Reported
DeepSeek V4.1 Flash on 4x 8 GB plus RTX PRO 6000 (ExLlamaV3 TP5)DeepSeek-V4.1-Flash-1017EXL3 3.5 bpw (LibertAIDAI; routed experts 4 bpw on layers 0-9 and 30-39, 3 bpw on 10-29; attention and shared experts 6 bpw) plus the official Engram shardsExLlamaV3 fork (anoane/exllamav3-anemone) + TabbyAPI fork (anoane/tabbyAPI)8 GB1,048,576 FP16Reported
DeepSeek V4.1 Flash on 6x 8 GB (TP6)DeepSeek-V4.1-Flash-1017Official checkpoint: dense MXFP8 through an SM80 Marlin W8A16 path, routed experts MXFP4 through W4A16; KV FP8Custom SM80 vLLM (patched from public commit 79a7108d9)8 GB1,048,576Reported
Qwen3.8 Flash Next on 2x 8 GB (vektorprime PP2)Qwen3.8-Flash-NextAWQ INT4: cyankiwi/Qwen3.8-Flash-Next-AWQ-INT4; 95 GiB BF16 PLE table in host RAM (FP8 table optional)vLLM nightly + PR #53899 (a5530b9) + the repo's patch scripts8 GB262,144Reported
Qwen3.8 Flash Next on 3x 8 GB (clee PP3)Qwen3.8-Flash-NextW4A16 AutoRound: Intel/Qwen3.8-Flash-Next-W4A16-AutoRound; 95.4 GiB BF16 PLE table streamed from NVMe; KV fp8 e4m3 (software-decoded on SM80)vLLM nightly 7d0b4e57 plus the author's pure-Python fork clee/vllm-sm80 (branch q38fn/stack)8 GB1,048,576 (YaRN x4)Proven
DeepSeek V4.1 Flash on 2x 8 GB (Project Maya)DeepSeek-V4.1-Flash-1017 (748B in the Maya README)Maya GGUF (Maya-S 340 GB or Maya-XS 277 GB; Engram tables stay on SSD)Project Maya (Strata and llama.cpp lineage)8 GBReported
GLM 5.3 Flash on 2x 8 GB (Project Maya)GLM-5.3-Flash (321B in the Maya README)Maya-M GGUF (116 GB), every expert in VRAM; Maya-S (96.5 GB) and Maya-S24 (94.7 GB) also reportedProject Maya (Strata and llama.cpp lineage, EXL3 loading, DFlash2 drafter)8 GBReported
GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split)GLM-5.3-Flash (320B MoE, 18B active)EXL3 3.05 bpw: turboderp/GLM-5.3-Flash-exl3ExLlamaV3 1.5.4 (built for SM80) with a small OpenAI-compatible server8 GB392,960 (profile q8_384k)Proven
GLM 5.3 on 8x 8 GB (vllm-170hx-glm5)GLM-5.3 (full 78-layer DSA MoE, not Flash)GLM-5.3-Int4-Int8Mix-AWQ-g64; drafter incoai/GLM-5.3-DFlash2Morrowmake/vllm-cmp170hx branch ampere-glm53 plus fail-closed source patches8 GB1,048,576 per requestProven
GLM 5.3 on 8x 8 GB (Int4-Int8Mix PP8)GLM-5.3 (full 78-layer, not Flash)GLM-5.3-Int4-Int8Mix (INT4 / INT8 mix); KV fp8_ds_mlavLLM (build not stated)8 GB1,048,576Reported
GLM 5.3 Flash on 3x 8 GBGLM-5.3-Flash (320B MoE, 18B active)W4A16, about 4.3 bpw, 163 GiB on diskMorrowmake/vllm-cmp170hx (via the glm53-flash-cmp170hx-recipe)8 GBabout 1,000,000 (8-bit KV cache)Reported
Retrieved from "https://cmp.wiki/w/index.php?title=All_recipes&oldid=423"