Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > card count: 4 & decode tps: 90-110 & status: Reported

Use the filters below to narrow your results.

page:
DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build) (1) · DeepSeek V4.1 Flash on 4x 8 GB plus RTX PRO 6000 (ExLlamaV3 TP5) (1) · GLM 5.3 Flash on 4x 8 GB (kaka86mm EXL3 + Marlin sidecar + DFlash2) (1)
model:
DeepSeek-V4.1-Flash-1017 (2) · GLM-5.3-Flash (320B MoE, 18B active, multimodal) (1)
quant:
EXL3 3.5 bpw (LibertAIDAI; routed experts 4 bpw on layers 0-9 and 30-39, 3 bpw on 10-29; attention and shared experts 6 bpw) plus the official Engram shards (1) · EXL3 4 bpw (Mia-AiLab/GLM-5.3-Flash-EXL3-TR3-4bpw, 164 GB) plus an offline-generated Marlin INT4 decode sidecar (151 GB) (1) · Hybrid: all 40 expert layers at 3 bits (LibertAI 2.9 bpw pack with layers 18-22 grafted from the 3.5 bpw pack), official FP8 dense tensors (1)
engine:
ExLlamaV3 fork (anoane/exllamav3-anemone) + TabbyAPI fork (anoane/tabbyAPI) (1) · GNmac-lab recipe image 1.2.0 (vLLM with sm_80 patches) plus three patches (1) · vLLM: hyd998877/vllm_170hx_glm53-flash-exl3-optimized with kaka86mm's overlay patches; exllamav3 1.4.8 (1)
engine version:
final production build v30 (numbers measured on v24); repo commit 218b47f (1) · GNmac-lab at 2ad5741ab1; image ds41-cmp170hx:1.2.0-fidelity (1) · tag recipe-09 (309ef6d1); TabbyAPI deepseek-v41 (ce250ff) (1)
cards:
8 GB (3)
card count:
4 (3)
topology:
PP4 (VLLM_PP_LAYER_PARTITION=13,11,11,10), DFlash2 k=7; PCIe Gen2 x4, no P2P (1) · TP4, P2P on (1) · TP5 across four CMP 170HX and one RTX PRO 6000 Blackwell (native backend, reduces through host memory); MTP draft on the PRO (1)
context:
1,048,576 FP16 (1) · 262,144 (1) · 524,288 (1)
decode tps:
96 (1) · 99.9 (1) · 105 (1)
prefill tps:
2,077 (1) · 2,116 (1) · 2,514 (1)
vram per card:
64 GB, --gpu-memory-utilization 0.93 (0.94 is the ceiling) (1) · 64 GiB (64,911 MiB usable), GPU_MEMORY_UTILIZATION 0.98 (1) · gpu_split 58 GB per CMP, 70 GB on the PRO (1)
power cap:
150 W per CMP (PRO 400 W) (1) · 200 W (2)
driver:
610.43.03 (open modules) (1) · NVIDIA open 610.43.02 from a cmpunlocker fork with a P2P patch (1) · NVIDIA open 610.57.04 patched with Morrowmake/cmpunlocker 1f2c498 (profile 8gb) (1)
unlock build:
cmpunlocker (64 GB) (1) · cmpunlocker fork with P2P (without Morrowmake's TRAP31 patch) (1) · Morrowmake/cmpunlocker 1f2c498; modprobe RmForceEnableGen2=1, RMPcieLinkSpeed=0x1, ForceP2P=0x11 (1)
status: (Click arrow to add another value)
tested by:
alanoo (1) · anoane (1) · dkpoulsen (1)
tested on:
September 19, 2026 (1) · October 20, 2026 (1) · October 27, 2026 (1)
source:
https://github.com/alanoo81/deepseek-v41-flash-4x-cmp170hx (1) · https://github.com/anoane/09-deepseek-v41-flash-1017-exl3-tp5-1m (1) · https://github.com/kaka86mm/glm53-flash-170hx (1)

Showing below up to 3 results in range #1 to #3.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"