Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > cards: 8 GB & tested by: nathang6866

Use the filters below to narrow your results.

page:
MiMo V2.6 Flash on 4x 8 GB (1) · Qwen3.8 Flash Next on 1x 8 GB (ElectroHobo 3 bpw) (1) · Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) (1)
model:
MiMo-V2.6-Flash-RL (309B MoE, 15B active; text, image, video and audio input) (1) · Qwen3.8-Flash-Next (125B MoE per the recipe) (2)
quant:
AutoRound mixed 2/3-bit (klee100, 2.998 effective bpw, 47 GiB on card) plus halt95's FP8 n-gram table in host RAM (1) · NVFP4 (ModelOpt mixed precision): ProCreations/MiMo-V2.6-Flash-RL-NVFP4, run as 4-bit weights with BF16 activations through Marlin (1) · W4A16: halt95/Qwen3.8-Flash-Next-W4A16-Merlin (AutoRound INT4 g128 experts, INT8 GDN projections, FP8 n-gram table) (1)
engine:
Morrowmake/vllm-cmp170hx image (Qwen EXL3 release) with patch files mounted read-only (1) · Morrowmake/vllm-cmp170hx image (tag qwen38-recipe-1.0.0) plus one mounted file (vLLM #58235 vision-sink fix) (1) · Morrowmake/vllm-cmp170hx image with patch files mounted read-only (1)
engine version:
recipe 0.2.1; engine 18305948ac (2) · recipe 0.3.1; engine 18305948ac (1)
cards: (Click arrow to add another value)
card count:
1 (1) · 2 (1) · 4 (1)
topology:
single card (1) · TP2 with whole-expert expert parallel (1) · TP4 (P2P on); expert parallel off (1)
context:
1,048,576 (1) · 262,144 (2)
decode tps:
274.9 (1) · 296.4 (1) · 349.7 (1)
prefill tps:
2,722 (1) · 4,243 (1) · 4,696 (1)
vram per card:
64 GiB, --gpu-memory-utilization 0.95 (fixed) (1) · 64 GiB, GPU_UTIL 0.95 (fixed) (1) · 64 GiB; weights 45.5 GiB plus 10 GiB KV per card; GPU_UTIL 0.94 (fixed) (1)
power cap:
180 W (3)
driver:
580 or newer (measured on 610.57.04) (2) · 580 or newer (measured on 610.57.04); open modules with HMM for the table-on-disk mode (1)
unlock build:
cmpunlocker (64 GiB, 74 SMs) (1) · cmpunlocker (64 GiB, 74 SMs) with a P2P-capable build (2)
status:
Proven (1) · Reported (2)
tested by: (Click arrow to add another value)
tested on:
October 18, 2026 (1) · October 23, 2026 (2)
source:
https://github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe (1) · https://github.com/ElectroHobo/qwen38-flash-next-3bpw-1x-cmp170hx-recipe (1) · https://github.com/ElectroHobo/qwen38-flash-next-w4a16-2x-cmp170hx-recipe (1)

Showing below up to 3 results in range #1 to #3.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"