Drilldown: Recipe
More actions
Use the filters below to narrow your results.
page:
model:
quant:
AutoRound mixed 2/3-bit (klee100, 2.998 effective bpw, 47 GiB on card) plus halt95's FP8 n-gram table in host RAM (1) ·
Hybrid: routed experts EXL3 (mostly 3 bpw, 2 bpw on layers 18-22) from LibertAI's 2.9 bpw pack; attention, shared experts, indexer and DSpark drafter official FP8; KV fp8_ds_mla (1) ·
Hybrid: routed experts EXL3 2.9 bpw (LibertAIDAI), everything else official FP8/BF16; KV BF16 (1) ·
Mixed INT4 AutoRound (Minachist/Qwen3.8-Flash-Next-INT4-Mixed-AutoRound, about 65 GB) with the full BF16 PLE table on NVMe (1) ·
NVFP4 (ModelOpt mixed precision): ProCreations/MiMo-V2.6-Flash-RL-NVFP4, run as 4-bit weights with BF16 activations through Marlin (1)
engine:
Morrowmake/vllm-cmp170hx (vLLM fork, container image) (1) ·
Morrowmake/vllm-cmp170hx image (tag qwen38-recipe-1.0.0) plus one mounted file (vLLM #58235 vision-sink fix) (1) ·
Morrowmake/vllm-cmp170hx image with patch files mounted read-only (1) ·
vLLM 0.29.1rc1 nightly plus the checkpoint author's patches (1) ·
vLLM with a pinned sm_80 patch stack (built locally from source) (1)
engine version:
cards:
topology:
context:
vram per card:
power cap:
driver:
unlock build:
status:
tested by:
tested on:
source: (Click arrow to add another value)
Showing below up to 5 results in range #1 to #5.