Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > decode tps: 160-200 & status: Reported

Use the filters below to narrow your results.

page:
DeepSeek V4.1 Flash on 6x 8 GB (Deepinfer-SM80 TP2 x PP3) (1) · Qwen3.8 27B on 1x 8 GB (W4A16 vLLM) (1) · Qwen3.8 Flash Next on 2x 8 GB (mannydelrio PP2) (1) · Qwen3.8 Flash Next on 2x 8 GB (Plean PP2) (1)
model:
DeepSeek-V4.1-Flash-1017 (1) · Qwen3.8-27B (dense) (1) · Qwen3.8-Flash-Next (1) · Qwen3.8-Flash-Next-Uncensored (refusal-removed, from orcarouter) (1)
quant:
AWQ W4A16 g32 (leoncca/Qwen3.8-Flash-Next-Uncensored-AWQ-g32); PLE table converted to INT4 and served from SSD (1) · Mixed INT4 AutoRound (Minachist/Qwen3.8-Flash-Next-INT4-Mixed-AutoRound, about 65 GB) with the full BF16 PLE table on NVMe (1) · Official checkpoint (FP8 dense, FP4 experts); software FP8 on sm_80 (1) · W4A16 (INT4 weights, Marlin kernels) (1)
engine:
vLLM (1) · vLLM 0.29.1rc1 nightly plus the checkpoint author's patches (1) · vLLM PR #56201 head (79a7108d) plus the two Deepinfer-SM80 patches and 30 drop-in modules (1) · wtdcode/vllm-backport 3142ded1 plus a two-file patch and two tinybeast PLE plugins (venv, no container) (1)
engine version:
None (2) · Deepinfer-SM80 2a828fc lineage; vLLM v0.28.1rc0 base (1) · vllm-backport 3142ded1 (1)
cards:
8 GB (4)
card count:
1 (1) · 2 (2) · 6 (1)
topology:
PP2 (layer partition 25,23), no P2P (1) · PP2, MTP 3, no P2P (1) · single card (1) · TP2 x PP3, partition 14,14,12 (1)
context:
1,048,576 (1) · 262,144 (1) · 262,144 (256K in the post) (1) · 262,144 native (YaRN to about 500k in the recipe's long variant) (1)
decode tps:
160.8 (1) · 170 (1) · 177.8 (1) · 197 (1)
prefill tps:
None (1) · 4,037 (1) · 5,800 (1) · 7,006 (1)
vram per card:
64 GB (2) · 64 GB, --gpu-memory-utilization 0.94, TB_TORCH_MEM_CAP_GIB=61.5 (1) · 64 GiB (1)
power cap:
None (3) · 180 W (1)
driver:
None (3) · nvidia-open 610.43.02 + cmpunlocker v0.5 or newer (1)
unlock build:
None (3) · cmpunlocker v0.5+ (sudo ./install.sh --profile=8gb --no-iommu) (1)
status: (Click arrow to add another value)
tested by:
HalfVulpes (1) · kellertreppe123 (1) · mrmannyatx (1) · plean. (1)
tested on:
October 5, 2026 (1) · October 7, 2026 (1) · October 15, 2026 (1) · October 28, 2026 (1)
source:
https://github.com/HalfVulpes/Deepinfer-SM80 (1) · https://github.com/mannydelrio/qwen38-flash-next-2x-cmp170hx (1) · https://www.bozocentral.com/2x-cmp170hx-qwen3-flash-next (1) · https://www.reddit.com/r/LocalLLM/comments/1w2f03e/qwen3827b_w4a16_on_one_64_gb_cmp_170hx_16070/ (1)

Showing below up to 4 results in range #1 to #4.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"