Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > decode tps: 70-150 & status: Proven

Use the filters below to narrow your results.

page:
DeepSeek V4 Flash 0731 on 4x 8 GB (allover326 PP4 DSpark) (1) · Qwen3.8 Flash Next on 1x 8 GB (iIIusi0n SSD PLE) (1) · Qwen3.8 Flash Next on 3x 8 GB (clee PP3) (1)
model:
DeepSeek-V4-Flash-0731 (1) · Qwen3.8-Flash-Next (2)
quant:
AutoRound 3 bpw + MTP: klee100/Qwen3.8-Flash-Next-AutoRound-3bpw-MTP; 95.37 GiB BF16 PLE table on SSD (1) · Native MXFP4 experts + FP8 attention (official deepseek-ai checkpoint, about 155.4 GiB); KV cache fp8 (1) · W4A16 AutoRound: Intel/Qwen3.8-Flash-Next-W4A16-AutoRound; 95.4 GiB BF16 PLE table streamed from NVMe; KV fp8 e4m3 (software-decoded on SM80) (1)
engine:
vLLM a5a30471 plus a 13-file downstream patch (native Linux AIO PLE reader) (1) · vLLM nightly 7d0b4e57 plus the author's pure-Python fork clee/vllm-sm80 (branch q38fn/stack) (1) · vLLM: haosdent/vllm branch dsv4-flash-a100 plus allover326's 7 patches (DSpark under PP, sm_80 prefill top-k fallback, logits row chunking) (1)
engine version:
0.29.1rc1.dev402+ga5a30471f.ple1 (1) · base c3046d1 (recommended since 2026-08-13; first released on f8ea5bb); repo commit 3dd2d88 (1) · q38fn/stack on nightly-7d0b4e57 (1)
cards:
8 GB (3)
card count:
1 (1) · 3 (1) · 4 (1)
topology:
PP3, no P2P (works at PP2 on two cards) (1) · PP4 (VLLM_PP_LAYER_PARTITION=12,12,12,7) with DSpark, 5 draft tokens; PCIe Gen2 x4, no P2P (1) · single card (1)
context:
1,047,736 verified one-shot; 1,002,852 over a 405-turn chat with DSV4_LOGITS_ROW_CHUNK=64 (1) · 1,048,576 (YaRN x4) (1) · 262,144 advertised (tested through 32K) (1)
decode tps:
74.4 (1) · 98.1 (1) · 110.99 (1)
prefill tps:
2,636 (1) · 4,831 (1) · 5,300 (1)
vram per card:
64 GB, --gpu-memory-utilization 0.85 (1) · 64 GiB, --gpu-memory-utilization 0.96 (1) · 64 GiB, gpu-memory-utilization 0.92 (1)
power cap:
None (1) · 180 W (2)
driver:
None (2) · 610.43.03 (1)
unlock build:
None (1) · cmpunlocker (64 GiB) (1) · cmpunlocker VRAM unlock (65,536 MiB per card) (1)
status: (Click arrow to add another value)
tested by:
c133 (1) · deejay9983 (1) · iIIusi0n (1)
tested on:
August 2026 (1) · September 2026 (1) · October 2026 (1)
source:
https://github.com/allover326/deepseek-v4-cmp170hx (1) · https://github.com/clee/qwen3.8-flash-next-3x170hx (1) · https://github.com/iIIusi0n/qwen3.8-flash-next-cmp170hx (1)

Showing below up to 3 results in range #1 to #3.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"