Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > card count: 8 & decode tps: 70-160 & status: Reported

Use the filters below to narrow your results.

page:
DeepSeek V4.1 Flash on 8x 8 GB (Deepinfer-SM80 TP4 x PP2) (1) · DeepSeek V4.1 Flash on 8x 8 GB (Zanooda PP8) (1) · GLM 5.3 Flash on 8x 8 GB (promisezackr PP8 NVFP4 DFlash2) (1)
model:
DeepSeek-V4.1-Flash-1017 (2) · GLM-5.3-Flash (1)
quant:
LibertAIDAI/GLM-5.3-Flash-NVFP4 (ModelOpt NVFP4, routed experts only, 181 GiB), run through Marlin W4A16 kernels; GLM-5.3-Flash DFlash2 drafter (5 sliding-window layers) (1) · Official checkpoint (FP8 dense, FP4 experts); software FP8 on sm_80 (1) · Official checkpoint as downloaded: MXFP4 routed experts as Marlin W4A16, MXFP8 dense as Marlin W8A16; KV fp8 (1)
engine:
vLLM PR #56201 head (79a7108d) plus a 20-patch sm_80 series; image zanooda/vllm-sm80-ds41f:v41-sm80 (1) · vLLM PR #56201 head (79a7108d) plus the two Deepinfer-SM80 patches and 30 drop-in modules (1) · vLLM: official vllm/vllm-openai:glm53-flash image plus promisezackr's 24 patches (SM8x base from 344303947/vllm170hx, pipeline parallel for GLM-5.3-Flash, DFlash2 under PP8) (1)
engine version:
Deepinfer-SM80, vLLM v0.28.1rc0 base (1) · recipe 31318fd; patches 0001-0020 (1) · v0.1.dev20051+g487ecf187 (the image's vLLM); repo commit 90ec72e (1)
cards:
8 GB (3)
card count:
8 (3)
topology:
PP8 (VLLM_PP_LAYER_PARTITION=7,5,7,5,7,5,7,2) with DFlash2, 7 draft tokens; PCIe Gen2 x4, no P2P; host 2x Xeon E5-2690 v4, 96 GB DDR4 (1) · PP8, partition 5,5,5,5,5,5,5,5 (1) · TP4 x PP2, partition 20,20 (1)
context:
1,048,576 (2) · 1,048,576 (KV pool 6.62M tokens at utilization 0.90, about 6.3 concurrent 1M requests) (1)
decode tps:
117 (1) · 123 (1) · 153.35 (1)
prefill tps:
None (1) · 3,463 (1) · 6,066 (1)
vram per card:
64 GB, --gpu-memory-utilization 0.90 (1) · 64 GB; weights 35.9 GB per rank (ranks 1-6), 37.6 GB rank 0, 46.3 GB rank 7 (1) · 64 GiB, --gpu-memory-utilization 0.95 (1)
power cap:
None (2) · 180 W (one card unstable at 250 W) (1)
driver:
None (3)
unlock build:
None (1) · cmpunlocker VRAM unlock (64 GB per card) (1) · cmpunlocker with the BAR1 P2P patches (64 GiB BAR1 per card) (1)
status: (Click arrow to add another value)
tested by:
HalfVulpes (1) · promisezackr (1) · Zanooda (1)
tested on:
August 2026 (1) · October 2026 (2)
source:
https://github.com/HalfVulpes/Deepinfer-SM80 (1) · https://github.com/promisezackr/glm53-flash-170hx-pp8 (1) · https://github.com/Zanooda/deepseek-v4.1-flash-cmp170hx (1)

Showing below up to 3 results in range #1 to #3.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"