Drilldown: Recipe
More actions
Use the filters below to narrow your results.
page:
model:
quant:
EXL3 3.05 bpw: turboderp/GLM-5.3-Flash-exl3; DFlash2 drafter converted to EXL3 6 bpw (1) ·
GLM-5.3-Int4-Int8Mix-AWQ-g64; drafter incoai/GLM-5.3-DFlash2 (1) ·
Hybrid: LibertAI EXL3 2.9 bpw routed experts (layers 0-39), official FP8/FP4 for attention, shared experts, head, draft and Engram; KV fp8_ds_mla (1) ·
Maya-M GGUF (116 GB), every expert in VRAM; Maya-S (96.5 GB) and Maya-S24 (94.7 GB) also reported (1) ·
NVFP4 main weights through Marlin (no native FP4 on sm_80), mixed FP8 and BF16 MTP weights, FP8 PLE table (about 51.2 GB) kept on NVMe (1)
engine:
ExLlamaV3 (patched, pinned) (1) ·
Morrowmake/vllm-cmp170hx (bind-mounted sm80 package) plus the author's tensor-core MoE kernels (1) ·
Morrowmake/vllm-cmp170hx branch ampere-glm53 plus fail-closed source patches (1) ·
Project Maya (Strata and llama.cpp lineage, EXL3 loading, DFlash2 drafter) (1) ·
Qwen-Flash-SM80 overlay on vllm/vllm-openai:qwen38-flash-next (digest-pinned), FlashInfer PCIe IPC AllReduce, GDS/cuFile over NVMe P2PDMA (1)
engine version:
cards:
topology:
engine-managed split with expert tiering across VRAM, RAM and NVMe (1) ·
PP8 (also PP9 and PP10), TP1 (1) ·
TEP2 (TP2 plus expert parallel), MTP 6; one card at PCIe Gen2 x16 (x16 mod), one at Gen2 x4; GPU P2P required (1) ·
TP2 over NCCL (GPU P2P), DFlash2 K7 plus hybrid drafting (1) ·
TP4, peer access off (NCCL_P2P_DISABLE=1), host all-reduce (1)
context:
decode tps:
prefill tps:
vram per card:
power cap:
unlock build:
status:
tested by:
tested on:
source: (Click arrow to add another value)
Showing below up to 5 results in range #1 to #5.