Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > card count: 2-4 & prefill tps: None

Use the filters below to narrow your results.

page:
DeepSeek V4 Flash 0731 on 3x 8 GB (PP3 DSpark) (1) · DeepSeek V4.1 Flash on 2x 8 GB (Project Maya) (1) · GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split) (1) · GLM 5.3 Flash on 3x 8 GB (1) · Qwen3.8 Flash Next on 2x 8 GB (vektorprime PP2) (1)
model:
DeepSeek-V4-Flash-0731 (1) · DeepSeek-V4.1-Flash-1017 (748B in the Maya README) (1) · GLM-5.3-Flash (320B MoE, 18B active) (2) · Qwen3.8-Flash-Next (1)
quant:
AWQ INT4: cyankiwi/Qwen3.8-Flash-Next-AWQ-INT4; 95 GiB BF16 PLE table in host RAM (FP8 table optional) (1) · EXL3 3.05 bpw: turboderp/GLM-5.3-Flash-exl3 (1) · Maya GGUF (Maya-S 340 GB or Maya-XS 277 GB; Engram tables stay on SSD) (1) · Native MXFP4 experts + FP8 attention (official deepseek-ai checkpoint, about 155.4 GiB); KV cache fp8 (1) · W4A16, about 4.3 bpw, 163 GiB on disk (1)
engine:
ExLlamaV3 1.5.4 (built for SM80) with a small OpenAI-compatible server (1) · Morrowmake/vllm-cmp170hx (via the glm53-flash-cmp170hx-recipe) (1) · Project Maya (Strata and llama.cpp lineage) (1) · vLLM nightly + PR #53899 (a5530b9) + the repo's patch scripts (1) · vLLM: haosdent/vllm branch dsv4-flash-a100 with a vendored copy of allover326's 7 patches (image dsv4-a100:devel) (1)
engine version:
None (1) · 0.1.dev1+ga5530b90c.d20260829 (1) · f8ea5bb-era base with the allover326 patch set as of mid-August 2026 (1) · Flun/glm53-flash-cmp170hx-exl3 at 5933112 (1) · not stated (1)
cards:
8 GB (5)
card count:
2 (3) · 3 (2)
topology:
engine-managed expert tiering across VRAM, host RAM and NVMe (1) · layer split (no tensor parallel), no P2P (1) · PP2 with PLE CPU offload (1) · PP3 (1) · PP3 (VLLM_PP_LAYER_PARTITION=15,15,13) with DSpark, 5 draft tokens; PCIe Gen2 x4, no P2P (1)
context:
None (1) · 262,144 (1) · 392,960 (profile q8_384k) (1) · 691,968 (largest stable context at --gpu-memory-utilization 0.965) (1) · about 1,000,000 (8-bit KV cache) (1)
decode tps:
None (1) · 60 (1) · 63 (1) · 75.4 (1) · 93 (1)
prefill tps:
None (5)
vram per card:
64 GB (1) · 64 GB (65,536 MiB), --gpu-memory-utilization 0.965 (1) · 64 GiB (1) · 64 GiB; weights PP0 39.48 GiB, PP1 45.53 GiB; --gpu-memory-utilization 0.9 (1) · about 58 GiB of 64 GiB (116.6 GiB resident across both) (1)
power cap:
None (4) · 100 W (1)
driver:
None (4) · 610.43.02 (nvidia-driver-610-open) (1)
unlock build:
None (4) · cmpunlocker 360acd7 (2026-08-06), 8 GB profile (1)
status:
Proven (1) · Reported (4)
tested by:
68x0 (1) · pcsv. (1) · shirogeek (1) · tim-odonnell (1) · vektor_prime (1)
tested on:
August 2026 (2) · October 2026 (3)
source:
https://discord.com/channels/1518634379604136178/1518695289278566450/1561743826945843304 (1) · https://github.com/Flun/glm53-flash-cmp170hx-exl3 (1) · https://github.com/mw00/project-maya#how-fast-is-it (1) · https://github.com/tim-odonnell/cmp170hx-deepseek-v4-flash (1) · https://github.com/vektorprime/qwen38-flash-next-pp2 (1)

Showing below up to 5 results in range #1 to #5.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"