Drilldown: Recipe
More actions
Recipe > decode tps:
70-90 
& source :
https://github.com/Morrowmake/dsv41-flash-cmp170hx-recipe or
Other
& source :
https://github.com/Morrowmake/dsv41-flash-cmp170hx-recipe or
Other Use the filters below to narrow your results.
page:
model:
quant:
AWQ INT4: cyankiwi/Qwen3.8-Flash-Next-AWQ-INT4; 95 GiB BF16 PLE table in host RAM (FP8 table optional) (1) ·
Official block FP8, about 126 GiB including the about 50 GiB FP8 n-gram (PLE) table, which is served from host RAM; KV cache bf16 (the QSA layers refuse fp8) (1) ·
Official checkpoint: dense MXFP8 through an SM80 Marlin W8A16 path, routed experts MXFP4 through W4A16; KV FP8 (1) ·
W4A16 AutoRound: Intel/Qwen3.8-Flash-Next-W4A16-AutoRound; 95.4 GiB BF16 PLE table streamed from NVMe; KV fp8 e4m3 (software-decoded on SM80) (1)
engine:
Custom SM80 vLLM (patched from public commit 79a7108d9) (1) ·
vLLM nightly + PR #53899 (a5530b9) + the repo's patch scripts (1) ·
vLLM nightly 7d0b4e57 plus the author's pure-Python fork clee/vllm-sm80 (branch q38fn/stack) (1) ·
vLLM: the vllm/vllm-openai:qwen38-flash-next image with zebgop-ops patches 0001 to 0016 (never 0008-WITHDRAWN) and three ported files (1)
engine version:
cards:
topology:
context:
prefill tps:
vram per card:
64 GB, --gpu-memory-utilization 0.95 (1) ·
64 GiB, gpu-memory-utilization 0.92 (1) ·
64 GiB; --gpu-memory-utilization 0.94 plus per-rank KV budgets; measured 34.0-34.3 GiB per 12-layer rank, 37.9 GiB on the drafter rank (1) ·
64 GiB; weights PP0 39.48 GiB, PP1 45.53 GiB; --gpu-memory-utilization 0.9 (1)
driver:
unlock build:
status:
tested by:
tested on:
source: (Click arrow to add another value)
Showing below up to 4 results in range #1 to #4.