Drilldown: Recipe
More actions
Use the filters below to narrow your results.
page:
model:
quant:
AWQ INT4: cyankiwi/Qwen3.8-Flash-Next-AWQ-INT4; 95 GiB BF16 PLE table in host RAM (FP8 table optional) (1) ·
EXL3 3.05 bpw: turboderp/GLM-5.3-Flash-exl3 (1) ·
GLM-5.3-Int4-Int8Mix-AWQ-g64; drafter incoai/GLM-5.3-DFlash2 (1) ·
Maya-M GGUF (116 GB), every expert in VRAM; Maya-S (96.5 GB) and Maya-S24 (94.7 GB) also reported (1) ·
Official checkpoint: dense MXFP8 through an SM80 Marlin W8A16 path, routed experts MXFP4 through W4A16; KV FP8 (1) ·
W4A16 AutoRound: Intel/Qwen3.8-Flash-Next-W4A16-AutoRound; 95.4 GiB BF16 PLE table streamed from NVMe; KV fp8 e4m3 (software-decoded on SM80) (1)
engine:
Custom SM80 vLLM (patched from public commit 79a7108d9) (1) ·
ExLlamaV3 1.5.4 (built for SM80) with a small OpenAI-compatible server (1) ·
Morrowmake/vllm-cmp170hx branch ampere-glm53 plus fail-closed source patches (1) ·
Project Maya (Strata and llama.cpp lineage, EXL3 loading, DFlash2 drafter) (1) ·
vLLM nightly + PR #53899 (a5530b9) + the repo's patch scripts (1) ·
vLLM nightly 7d0b4e57 plus the author's pure-Python fork clee/vllm-sm80 (branch q38fn/stack) (1)
engine version: (Click arrow to add another value)
cards:
topology:
context:
decode tps:
vram per card:
power cap:
driver:
unlock build:
status:
tested by:
tested on:
source: (Click arrow to add another value)
Showing below up to 6 results in range #1 to #6.