Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > cards: 8 GB & prefill tps: 0-1800

Use the filters below to narrow your results.

page:
DeepSeek V4.1 Flash on 4x 8 GB (CurtisAccelerate TC-MoE) (1) · GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 TP2) (1) · GLM 5.3 Flash on 2x 8 GB (Project Maya) (1)
model:
DeepSeek-V4.1-Flash-1017 (1) · GLM-5.3-Flash (320B MoE, 18B active) (1) · GLM-5.3-Flash (321B in the Maya README) (1)
quant:
EXL3 3.05 bpw: turboderp/GLM-5.3-Flash-exl3; DFlash2 drafter converted to EXL3 6 bpw (1) · Hybrid: LibertAI EXL3 2.9 bpw routed experts (layers 0-39), official FP8/FP4 for attention, shared experts, head, draft and Engram; KV fp8_ds_mla (1) · Maya-M GGUF (116 GB), every expert in VRAM; Maya-S (96.5 GB) and Maya-S24 (94.7 GB) also reported (1)
engine:
ExLlamaV3 (patched, pinned) (1) · Morrowmake/vllm-cmp170hx (bind-mounted sm80 package) plus the author's tensor-core MoE kernels (1) · Project Maya (Strata and llama.cpp lineage, EXL3 loading, DFlash2 drafter) (1)
engine version:
upstream dev 0f685aa9fd plus 9 patches; repo db01c256 (1) · v1.0.6 (this row) (1) · vllm-cmp170hx 18305948ac; TC-MoE fb9786ec (1)
cards: (Click arrow to add another value)
card count:
2 (2) · 4 (1)
topology:
engine-managed split with expert tiering across VRAM, RAM and NVMe (1) · TP2 over NCCL (GPU P2P), DFlash2 K7 plus hybrid drafting (1) · TP4, peer access off (NCCL_P2P_DISABLE=1), host all-reduce (1)
context:
None (1) · 262,144 (weights address 1,048,576) (1) · 392,960 per request; 655,360-token Q8 KV pool shared by two requests (1)
decode tps:
61.3 (1) · 107.67 (1) · 144.2 (1)
prefill tps:
514 (1) · 1,523 (1) · 1,533 (1)
vram per card:
64 GB (1) · about 62.5 GB used of 64 GB, --gpu-memory-utilization 0.97 (1) · about 62.9 / 62.3 GiB idle, peaks about 64.4 / 63.0 GiB with vision (1)
power cap:
None (1) · 140 W (1) · 180 W (author's cards); about 170 W with a modest memory OC in his Discord numbers (1)
driver:
None (2) · 610.57.04 (1)
unlock build:
None (2) · cmpunlocker (64 GiB); P2P-capable driver for the NCCL backend (1)
status:
Reported (3)
tested by:
CurtisAccelerate (1) · nestaa51 (1) · ZackO2o (1)
tested on:
October 13, 2026 (1) · October 22, 2026 (1)
source:
https://github.com/alexvnesta/glm53-flash-cmp170hx-tp (1) · https://github.com/CurtisAccelerate/DeepSeekv41-Flash-1017-EXL3-CMP170HX (1) · https://github.com/mw00/project-maya#how-fast-is-it (1)

Showing below up to 3 results in range #1 to #3.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"