Drilldown: Recipe
More actions
Use the filters below to narrow your results.
page:
model:
quant:
AutoRound mixed 2/3-bit (klee100, 2.998 effective bpw, 47 GiB on card) plus halt95's FP8 n-gram table in host RAM (1) ·
Hybrid: routed experts EXL3 (mostly 3 bpw, 2 bpw on layers 18-22) from LibertAI's 2.9 bpw pack; attention, shared experts, indexer and DSpark drafter official FP8; KV fp8_ds_mla (1) ·
NVFP4 (ModelOpt mixed precision): ProCreations/MiMo-V2.6-Flash-RL-NVFP4, run as 4-bit weights with BF16 activations through Marlin (1) ·
W4A16: halt95/Qwen3.8-Flash-Next-W4A16-Merlin (AutoRound INT4 g128 experts, INT8 GDN projections, FP8 n-gram table) (1)
engine:
Morrowmake/vllm-cmp170hx image (Qwen EXL3 release) with patch files mounted read-only (1) ·
Morrowmake/vllm-cmp170hx image (tag qwen38-recipe-1.0.0) plus one mounted file (vLLM #58235 vision-sink fix) (1) ·
Morrowmake/vllm-cmp170hx image with patch files mounted read-only (1) ·
vLLM with a pinned sm_80 patch stack (built locally from source) (1)
engine version:
cards: (Click arrow to add another value)
topology:
context:
vram per card:
power cap:
driver:
unlock build:
status:
tested by: (Click arrow to add another value)
tested on:
source:
Showing below up to 4 results in range #1 to #4.