Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > decode tps: 80-110 & status: Reported

Use the filters below to narrow your results.

page:
DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build) (1) · DeepSeek V4.1 Flash on 4x 8 GB plus RTX PRO 6000 (ExLlamaV3 TP5) (1) · GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 TP2) (1)
model:
DeepSeek-V4.1-Flash-1017 (2) · GLM-5.3-Flash (320B MoE, 18B active) (1)
quant:
EXL3 3.05 bpw: turboderp/GLM-5.3-Flash-exl3; DFlash2 drafter converted to EXL3 6 bpw (1) · EXL3 3.5 bpw (LibertAIDAI; routed experts 4 bpw on layers 0-9 and 30-39, 3 bpw on 10-29; attention and shared experts 6 bpw) plus the official Engram shards (1) · Hybrid: all 40 expert layers at 3 bits (LibertAI 2.9 bpw pack with layers 18-22 grafted from the 3.5 bpw pack), official FP8 dense tensors (1)
engine:
ExLlamaV3 (patched, pinned) (1) · ExLlamaV3 fork (anoane/exllamav3-anemone) + TabbyAPI fork (anoane/tabbyAPI) (1) · GNmac-lab recipe image 1.2.0 (vLLM with sm_80 patches) plus three patches (1)
engine version:
GNmac-lab at 2ad5741ab1; image ds41-cmp170hx:1.2.0-fidelity (1) · tag recipe-09 (309ef6d1); TabbyAPI deepseek-v41 (ce250ff) (1) · upstream dev 0f685aa9fd plus 9 patches; repo db01c256 (1)
cards:
8 GB (3)
card count:
2 (1) · 4 (2)
topology:
TP2 over NCCL (GPU P2P), DFlash2 K7 plus hybrid drafting (1) · TP4, P2P on (1) · TP5 across four CMP 170HX and one RTX PRO 6000 Blackwell (native backend, reduces through host memory); MTP draft on the PRO (1)
context:
1,048,576 FP16 (1) · 262,144 (1) · 392,960 per request; 655,360-token Q8 KV pool shared by two requests (1)
decode tps:
99.9 (1) · 105 (1) · 107.67 (1)
prefill tps:
1,533 (1) · 2,116 (1) · 2,514 (1)
vram per card:
64 GiB (64,911 MiB usable), GPU_MEMORY_UTILIZATION 0.98 (1) · about 62.9 / 62.3 GiB idle, peaks about 64.4 / 63.0 GiB with vision (1) · gpu_split 58 GB per CMP, 70 GB on the PRO (1)
power cap:
150 W per CMP (PRO 400 W) (1) · 180 W (author's cards); about 170 W with a modest memory OC in his Discord numbers (1) · 200 W (1)
driver:
610.57.04 (1) · NVIDIA open 610.43.02 from a cmpunlocker fork with a P2P patch (1) · NVIDIA open 610.57.04 patched with Morrowmake/cmpunlocker 1f2c498 (profile 8gb) (1)
unlock build:
cmpunlocker (64 GiB); P2P-capable driver for the NCCL backend (1) · cmpunlocker fork with P2P (without Morrowmake's TRAP31 patch) (1) · Morrowmake/cmpunlocker 1f2c498; modprobe RmForceEnableGen2=1, RMPcieLinkSpeed=0x1, ForceP2P=0x11 (1)
status: (Click arrow to add another value)
tested by:
alanoo (1) · anoane (1) · nestaa51 (1)
tested on:
October 13, 2026 (1) · October 20, 2026 (1) · October 27, 2026 (1)
source:
https://github.com/alanoo81/deepseek-v41-flash-4x-cmp170hx (1) · https://github.com/alexvnesta/glm53-flash-cmp170hx-tp (1) · https://github.com/anoane/09-deepseek-v41-flash-1017-exl3-tp5-1m (1)

Showing below up to 3 results in range #1 to #3.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"