Drilldown: Recipe
More actions
Use the filters below to narrow your results.
page:
model:
quant:
AWQ INT4: cyankiwi/Qwen3.8-Flash-Next-AWQ-INT4; 95 GiB BF16 PLE table in host RAM (FP8 table optional) (1) ·
LibertAIDAI/GLM-5.3-Flash-NVFP4 (ModelOpt NVFP4, routed experts only, 181 GiB), run through Marlin W4A16 kernels; GLM-5.3-Flash DFlash2 drafter (5 sliding-window layers) (1) ·
Native MXFP4 experts + FP8 attention (official deepseek-ai checkpoint, about 155.4 GiB); KV cache fp8 (1) ·
Official checkpoint: dense MXFP8 through an SM80 Marlin W8A16 path, routed experts MXFP4 through W4A16; KV FP8 (1)
engine:
Custom SM80 vLLM (patched from public commit 79a7108d9) (1) ·
vLLM nightly + PR #53899 (a5530b9) + the repo's patch scripts (1) ·
vLLM: haosdent/vllm branch dsv4-flash-a100 with a vendored copy of allover326's 7 patches (image dsv4-a100:devel) (1) ·
vLLM: official vllm/vllm-openai:glm53-flash image plus promisezackr's 24 patches (SM8x base from 344303947/vllm170hx, pipeline parallel for GLM-5.3-Flash, DFlash2 under PP8) (1)
engine version:
cards:
topology:
PP2 with PLE CPU offload (1) ·
PP3 (VLLM_PP_LAYER_PARTITION=15,15,13) with DSpark, 5 draft tokens; PCIe Gen2 x4, no P2P (1) ·
PP8 (VLLM_PP_LAYER_PARTITION=7,5,7,5,7,5,7,2) with DFlash2, 7 draft tokens; PCIe Gen2 x4, no P2P; host 2x Xeon E5-2690 v4, 96 GB DDR4 (1) ·
TP6 (virtual-head padding), PP1 (1)
context:
prefill tps:
vram per card:
unlock build:
status:
tested by:
tested on:
Showing below up to 4 results in range #1 to #4.