Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > decode tps: 140-500 & status: Proven

Use the filters below to narrow your results.

page:
DeepSeek V4.1 Flash on 4x 8 GB (GNmac) (1) · GLM 5.3 Flash on 4x 8 GB (1) · Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) (1)
model:
DeepSeek-V4.1-Flash-1017 (1) · GLM-5.3-Flash (320B MoE, 18B active) (1) · Qwen3.8-Flash-Next (125B MoE per the recipe) (1)
quant:
Hybrid: routed experts EXL3 (mostly 3 bpw, 2 bpw on layers 18-22) from LibertAI's 2.9 bpw pack; attention, shared experts, indexer and DSpark drafter official FP8; KV fp8_ds_mla (1) · W4A16 (INT4 weights, group size 128, FP16 activations): canada-quant/GLM-5.3-Flash-W4A16-MTP (1) · W4A16: halt95/Qwen3.8-Flash-Next-W4A16-Merlin (AutoRound INT4 g128 experts, INT8 GDN projections, FP8 n-gram table) (1)
engine:
Morrowmake/vllm-cmp170hx (vLLM fork, container image) (1) · Morrowmake/vllm-cmp170hx image (Qwen EXL3 release) with patch files mounted read-only (1) · vLLM with a pinned sm_80 patch stack (built locally from source) (1)
engine version:
recipe 0.3.1; engine 18305948ac (1) · recipe 1.2.0 (1) · recipe 1.8.0, engine 819146023f (1)
cards:
8 GB (3)
card count:
2 (1) · 4 (2)
topology:
TP2 with whole-expert expert parallel (1) · TP4 (default) or PP4 (1) · TP4 (default) or PP4 for x1/x4 links (1)
context:
262,144 (2) · 262,144 (up to 1,048,576) (1)
decode tps:
258 (1) · 349.7 (1) · 442.1 (1)
prefill tps:
2,416 (1) · 3,161 (1) · 4,696 (1)
vram per card:
64 GiB exposed, --gpu-memory-utilization 0.95 (1) · 64 GiB, GPU_MEMORY_UTILIZATION 0.95 (1) · 64 GiB, GPU_UTIL 0.95 (fixed) (1)
power cap:
180 W (2) · 180 W (nvidia-smi -pl 180) (1)
driver:
580 or newer (measured on 610.57.04); open modules with HMM for the table-on-disk mode (1) · CUDA 13 capable; measured on 610.43.02 (1) · NVIDIA 580 or newer; P2P needs the Morrowmake/cmpunlocker build (1)
unlock build:
cmpunlocker (64 GiB, 74 SMs) with a P2P-capable build (1) · cmpunlocker (64 GiB, 74 SMs) with a P2P-capable driver for TP4 (1) · cmpunlocker (64 GiB, 74 SMs); Morrowmake/cmpunlocker for P2P (1)
status: (Click arrow to add another value)
tested by:
greencandle9753 (1) · nathang6866 (1) · voidgiraffe (1)
tested on:
October 16, 2026 (1) · October 23, 2026 (1) · October 26, 2026 (1)
source:
https://github.com/ElectroHobo/qwen38-flash-next-w4a16-2x-cmp170hx-recipe (1) · https://github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx (1) · https://github.com/Morrowmake/glm53-flash-cmp170hx-recipe (1)

Showing below up to 3 results in range #1 to #3.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"