Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > decode tps: 70-100 & status: Reported

Use the filters below to narrow your results.

page:
DeepSeek V4.1 Flash on 4x 8 GB plus RTX PRO 6000 (ExLlamaV3 TP5) (1) · DeepSeek V4.1 Flash on 6x 8 GB (TP6) (1) · Qwen3.8 Flash Next on 2x 8 GB (vektorprime PP2) (1)
model:
DeepSeek-V4.1-Flash-1017 (2) · Qwen3.8-Flash-Next (1)
quant:
AWQ INT4: cyankiwi/Qwen3.8-Flash-Next-AWQ-INT4; 95 GiB BF16 PLE table in host RAM (FP8 table optional) (1) · EXL3 3.5 bpw (LibertAIDAI; routed experts 4 bpw on layers 0-9 and 30-39, 3 bpw on 10-29; attention and shared experts 6 bpw) plus the official Engram shards (1) · Official checkpoint: dense MXFP8 through an SM80 Marlin W8A16 path, routed experts MXFP4 through W4A16; KV FP8 (1)
engine:
Custom SM80 vLLM (patched from public commit 79a7108d9) (1) · ExLlamaV3 fork (anoane/exllamav3-anemone) + TabbyAPI fork (anoane/tabbyAPI) (1) · vLLM nightly + PR #53899 (a5530b9) + the repo's patch scripts (1)
engine version:
0.1.dev1+ga5530b90c.d20260829 (1) · reports 0.0.0; patches/public-base-to-runtime.patch (109 files) (1) · tag recipe-09 (309ef6d1); TabbyAPI deepseek-v41 (ce250ff) (1)
cards:
8 GB (3)
card count:
2 (1) · 4 (1) · 6 (1)
topology:
PP2 with PLE CPU offload (1) · TP5 across four CMP 170HX and one RTX PRO 6000 Blackwell (native backend, reduces through host memory); MTP draft on the PRO (1) · TP6 (virtual-head padding), PP1 (1)
context:
1,048,576 (1) · 1,048,576 FP16 (1) · 262,144 (1)
decode tps:
75.4 (1) · 78.55 (1) · 99.9 (1)
prefill tps:
None (2) · 2,116 (1)
vram per card:
64 GB, --gpu-memory-utilization 0.95 (1) · 64 GiB; weights PP0 39.48 GiB, PP1 45.53 GiB; --gpu-memory-utilization 0.9 (1) · gpu_split 58 GB per CMP, 70 GB on the PRO (1)
power cap:
None (2) · 150 W per CMP (PRO 400 W) (1)
driver:
None (2) · NVIDIA open 610.57.04 patched with Morrowmake/cmpunlocker 1f2c498 (profile 8gb) (1)
unlock build:
None (2) · Morrowmake/cmpunlocker 1f2c498; modprobe RmForceEnableGen2=1, RMPcieLinkSpeed=0x1, ForceP2P=0x11 (1)
status: (Click arrow to add another value)
tested by:
anoane (1) · vektor_prime (1) · wgkbfcp (1)
tested on:
August 2026 (1) · October 2026 (2)
source:
https://github.com/anoane/09-deepseek-v41-flash-1017-exl3-tp5-1m (1) · https://github.com/vektorprime/qwen38-flash-next-pp2 (1) · https://github.com/wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe (1)

Showing below up to 3 results in range #1 to #3.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"