Drilldown: Recipe - CMP Wiki

Jump to content

Choose a table:

Recipe > cards: 8 GB

& prefill tps: 3200-5000

Use the filters below to narrow your results.

DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake) (1) · DeepSeek V4.1 Flash on 6x 8 GB (Deepinfer-SM80 TP2 x PP3) (1) · DeepSeek V4.1 Flash on 8x 8 GB (Deepinfer-SM80 TP4 x PP2) (1) · Qwen3.8 Flash Next on 1x 8 GB (ElectroHobo 3 bpw) (1) · Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) (1) · Qwen3.8 Flash Next on 3x 8 GB (clee PP3) (1)

DeepSeek-V4.1-Flash-1017 (3) · Qwen3.8-Flash-Next (1) · Qwen3.8-Flash-Next (125B MoE per the recipe) (2)

AutoRound mixed 2/3-bit (klee100, 2.998 effective bpw, 47 GiB on card) plus halt95's FP8 n-gram table in host RAM (1) · Hybrid: routed experts EXL3 2.9 bpw (LibertAIDAI), everything else official FP8/BF16; KV BF16 (1) · Official checkpoint (FP8 dense, FP4 experts); software FP8 on sm_80 (2) · W4A16 AutoRound: Intel/Qwen3.8-Flash-Next-W4A16-AutoRound; 95.4 GiB BF16 PLE table streamed from NVMe; KV fp8 e4m3 (software-decoded on SM80) (1) · W4A16: halt95/Qwen3.8-Flash-Next-W4A16-Merlin (AutoRound INT4 g128 experts, INT8 GDN projections, FP8 n-gram table) (1)

Morrowmake/vllm-cmp170hx (vLLM fork, container image) (1) · Morrowmake/vllm-cmp170hx image (Qwen EXL3 release) with patch files mounted read-only (1) · Morrowmake/vllm-cmp170hx image with patch files mounted read-only (1) · vLLM nightly 7d0b4e57 plus the author's pure-Python fork clee/vllm-sm80 (branch q38fn/stack) (1) · vLLM PR #56201 head (79a7108d) plus the two Deepinfer-SM80 patches and 30 drop-in modules (2)

Deepinfer-SM80 2a828fc lineage; vLLM v0.28.1rc0 base (1) · Deepinfer-SM80, vLLM v0.28.1rc0 base (1) · q38fn/stack on nightly-7d0b4e57 (1) · recipe 0.2.1; engine 18305948ac (1) · recipe 0.3.1; engine 18305948ac (1) · recipe 1.0.0, engine a0c57b5237 (1)

PP3, no P2P (works at PP2 on two cards) (1) · single card (1) · TP2 with whole-expert expert parallel (1) · TP2 x PP3, partition 14,14,12 (1) · TP4 with expert parallel (96 whole experts per card) (1) · TP4 x PP2, partition 20,20 (1)

1,048,576 (2) · 1,048,576 (YaRN x4) (1) · 262,144 (3)

3,000 - 5,000 (6)

64 GiB (1) · 64 GiB exposed, --gpu-memory-utilization 0.95 (1) · 64 GiB, --gpu-memory-utilization 0.95 (1) · 64 GiB, --gpu-memory-utilization 0.95 (fixed) (1) · 64 GiB, gpu-memory-utilization 0.92 (1) · 64 GiB, GPU_UTIL 0.95 (fixed) (1)

None (3) · 180 W (3)

None (3) · 580 or newer (measured on 610.57.04) (1) · 580 or newer (measured on 610.57.04); open modules with HMM for the table-on-disk mode (1) · NVIDIA 580 or newer; P2P needs the Morrowmake/cmpunlocker build (1)

None (3) · cmpunlocker (64 GiB, 74 SMs) (1) · cmpunlocker (64 GiB, 74 SMs) with a P2P-capable build (1) · cmpunlocker (64 GiB, 74 SMs); Morrowmake/cmpunlocker for P2P (1)

Proven (2) · Reported (4)

c133 (1) · HalfVulpes (2) · nathang6866 (2) · voidgiraffe (1)

October 10, 2026 (1) · October 23, 2026 (2) · October 24, 2026 (1) · October 28, 2026 (2)

https://github.com/clee/qwen3.8-flash-next-3x170hx (1) · https://github.com/ElectroHobo/qwen38-flash-next-3bpw-1x-cmp170hx-recipe (1) · https://github.com/ElectroHobo/qwen38-flash-next-w4a16-2x-cmp170hx-recipe (1) · https://github.com/HalfVulpes/Deepinfer-SM80 (2) · https://github.com/Morrowmake/dsv41-flash-cmp170hx-recipe (1)

Showing below up to 6 results in range #1 to #6.

D

DeepSeek V4.1 Flash on 8x 8 GB (Deepinfer-SM80 TP4 x PP2)

Q

Qwen3.8 Flash Next on 1x 8 GB (ElectroHobo 3 bpw)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"