Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > decode tps: 20-110 & status: Proven

Use the filters below to narrow your results.

page:
DeepSeek V4 Flash 0731 on 4x 8 GB (allover326 PP4 DSpark) (1) · GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split) (1) · GLM 5.3 on 8x 8 GB (vllm-170hx-glm5) (1) · Qwen3.8 Flash Next on 3x 8 GB (clee PP3) (1)
model:
DeepSeek-V4-Flash-0731 (1) · GLM-5.3 (full 78-layer DSA MoE, not Flash) (1) · GLM-5.3-Flash (320B MoE, 18B active) (1) · Qwen3.8-Flash-Next (1)
quant:
EXL3 3.05 bpw: turboderp/GLM-5.3-Flash-exl3 (1) · GLM-5.3-Int4-Int8Mix-AWQ-g64; drafter incoai/GLM-5.3-DFlash2 (1) · Native MXFP4 experts + FP8 attention (official deepseek-ai checkpoint, about 155.4 GiB); KV cache fp8 (1) · W4A16 AutoRound: Intel/Qwen3.8-Flash-Next-W4A16-AutoRound; 95.4 GiB BF16 PLE table streamed from NVMe; KV fp8 e4m3 (software-decoded on SM80) (1)
engine:
ExLlamaV3 1.5.4 (built for SM80) with a small OpenAI-compatible server (1) · Morrowmake/vllm-cmp170hx branch ampere-glm53 plus fail-closed source patches (1) · vLLM nightly 7d0b4e57 plus the author's pure-Python fork clee/vllm-sm80 (branch q38fn/stack) (1) · vLLM: haosdent/vllm branch dsv4-flash-a100 plus allover326's 7 patches (DSpark under PP, sm_80 prefill top-k fallback, logits row chunking) (1)
engine version:
base c3046d1 (recommended since 2026-08-13; first released on f8ea5bb); repo commit 3dd2d88 (1) · Flun/glm53-flash-cmp170hx-exl3 at 5933112 (1) · q38fn/stack on nightly-7d0b4e57 (1) · vLLM 0.30.1rc1, fork commit 378c37b0 (1)
cards:
8 GB (4)
card count:
2 (1) · 3 (1) · 4 (1) · 8 (1)
topology:
layer split (no tensor parallel), no P2P (1) · PP3, no P2P (works at PP2 on two cards) (1) · PP4 (VLLM_PP_LAYER_PARTITION=12,12,12,7) with DSpark, 5 draft tokens; PCIe Gen2 x4, no P2P (1) · PP8 (also PP9 and PP10), TP1 (1)
context:
1,047,736 verified one-shot; 1,002,852 over a 405-turn chat with DSV4_LOGITS_ROW_CHUNK=64 (1) · 1,048,576 (YaRN x4) (1) · 1,048,576 per request (1) · 392,960 (profile q8_384k) (1)
decode tps:
48.7 (1) · 60 (1) · 74.4 (1) · 98.1 (1)
prefill tps:
None (1) · 1,833 (1) · 4,831 (1) · 5,300 (1)
vram per card:
64 GB, --gpu-memory-utilization 0.85 (1) · 64 GiB, gpu-memory-utilization 0.92 (1) · 64 GiB, GPU_UTIL 0.96 at PP8 (1) · about 58 GiB of 64 GiB (116.6 GiB resident across both) (1)
power cap:
None (2) · 170 W (author's 10-card run) (1) · 180 W (1)
driver:
None (4)
unlock build:
None (3) · cmpunlocker VRAM unlock (65,536 MiB per card) (1)
status: (Click arrow to add another value)
tested by:
68x0 (1) · c133 (1) · deejay9983 (1) · reala7096 (1)
tested on:
August 2026 (1) · September 2026 (1) · October 2026 (2)
source:
https://github.com/allover326/deepseek-v4-cmp170hx (1) · https://github.com/arvkmr/vllm-170hx-glm5 (1) · https://github.com/clee/qwen3.8-flash-next-3x170hx (1) · https://github.com/Flun/glm53-flash-cmp170hx-exl3 (1)

Showing below up to 4 results in range #1 to #4.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"