Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > decode tps: 150-170 & status: Reported

Use the filters below to narrow your results.

page:
DeepSeek V4.1 Flash heretic on 4x 8 GB (1) · DeepSeek V4.1 Flash on 6x 8 GB (Deepinfer-SM80 TP2 x PP3) (1) · DeepSeek V4.1 Flash on 8x 8 GB (Deepinfer-SM80 TP4 x PP2) (1) · Qwen3.8 Flash Next on 2x 8 GB (Morrowmake EXL3) (1) · Qwen3.8 Flash Next on 2x 8 GB (Qwen-Flash-SM80, GDS TEP2) (1)
model:
DeepSeek-V4.1-Flash-1017 (2) · DeepSeek-V4.1-Flash-1017 heretic (pqhaz abliteration) (1) · Qwen3.8-Flash-Next (1) · Qwen3.8-Flash-Next (nvidia/Qwen3.8-Flash-Next-NVFP4, revision fc694b54, 11 shards, 132.68 GB) (1)
quant:
EXL3 5.05 bpw; PLE (n-gram) table read from SSD (1) · Hybrid: routed experts EXL3 2.9 bpw (LibertAIDAI), everything else official FP8/BF16 with the 38 heretic attn.wo_b tensors (1) · NVFP4 main weights through Marlin (no native FP4 on sm_80), mixed FP8 and BF16 MTP weights, FP8 PLE table (about 51.2 GB) kept on NVMe (1) · Official checkpoint (FP8 dense, FP4 experts); software FP8 on sm_80 (2)
engine:
Morrowmake/vllm-cmp170hx (EXL3 path) (1) · Morrowmake/vllm-cmp170hx (vLLM fork, container image) (1) · Qwen-Flash-SM80 overlay on vllm/vllm-openai:qwen38-flash-next (digest-pinned), FlashInfer PCIe IPC AllReduce, GDS/cuFile over NVMe P2PDMA (1) · vLLM PR #56201 head (79a7108d) plus the two Deepinfer-SM80 patches and 30 drop-in modules (2)
engine version:
Deepinfer-SM80 2a828fc lineage; vLLM v0.28.1rc0 base (1) · Deepinfer-SM80, vLLM v0.28.1rc0 base (1) · recipe 1.0.0 (commit d4d1f525), engine a0c57b52 (1) · tag qwen38-recipe-1.0.0, engine 18305948ac (1) · v0.3.0 (commit 4d229f1, 2026-10-14); base vLLM 0.1.dev20073+g8e685d198 (1)
cards:
8 GB (5)
card count:
2 (2) · 4 (1) · 6 (1) · 8 (1)
topology:
TEP2 (TP2 plus expert parallel), MTP 6; one card at PCIe Gen2 x16 (x16 mod), one at Gen2 x4; GPU P2P required (1) · TP2 with expert parallel (1) · TP2 x PP3, partition 14,14,12 (1) · TP4 (1) · TP4 x PP2, partition 20,20 (1)
context:
1,048,576 (2) · 262,144 (3)
decode tps:
152.6 (1) · 153.35 (1) · 158 (1) · 160.8 (1) · 168.29 (1)
prefill tps:
1,986.38 (1) · 2,453 (1) · 2,651 (1) · 3,463 (1) · 4,037 (1)
vram per card:
64 GB, --gpu-memory-utilization 0.93 (v0.3.0) (1) · 64 GiB (1) · 64 GiB, --gpu-memory-utilization 0.92 (miner co-resident) (1) · 64 GiB, --gpu-memory-utilization 0.95 (2)
power cap:
None (5)
driver:
None (4) · NVIDIA open 610.43.03, CUDA 13.0 (1)
unlock build:
None (4) · bayley/cmpunlocker 5a7bb4b, sudo ./install.sh --p2p --no-iommu (1)
status: (Click arrow to add another value)
tested by:
darkmaniac7 (1) · HalfVulpes (2) · nguyenthimy2022kg-alt (1) · voidgiraffe (1)
tested on:
September 8, 2026 (1) · October 16, 2026 (1) · October 28, 2026 (2) · October 29, 2026 (1)
source:
https://github.com/HalfVulpes/Deepinfer-SM80 (2) · https://github.com/Morrowmake/dsv41-flash-cmp170hx-recipe (1) · https://github.com/Morrowmake/qwen38-flash-next-cmp170hx-recipe (1) · https://github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX (1)

Showing below up to 5 results in range #1 to #5.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"