Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Drilldown: Recipe

More actions
Recipe > decode tps: 20-80 & prefill tps: None

Use the filters below to narrow your results.

page:
DeepSeek V4.1 Flash on 2x 8 GB (Project Maya) (1) · DeepSeek V4.1 Flash on 6x 8 GB (TP6) (1) · GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split) (1) · GLM 5.3 on 8x 8 GB (Int4-Int8Mix PP8) (1) · Qwen3.8 Flash Next on 2x 8 GB (vektorprime PP2) (1)
model:
DeepSeek-V4.1-Flash-1017 (1) · DeepSeek-V4.1-Flash-1017 (748B in the Maya README) (1) · GLM-5.3 (full 78-layer, not Flash) (1) · GLM-5.3-Flash (320B MoE, 18B active) (1) · Qwen3.8-Flash-Next (1)
quant:
AWQ INT4: cyankiwi/Qwen3.8-Flash-Next-AWQ-INT4; 95 GiB BF16 PLE table in host RAM (FP8 table optional) (1) · EXL3 3.05 bpw: turboderp/GLM-5.3-Flash-exl3 (1) · GLM-5.3-Int4-Int8Mix (INT4 / INT8 mix); KV fp8_ds_mla (1) · Maya GGUF (Maya-S 340 GB or Maya-XS 277 GB; Engram tables stay on SSD) (1) · Official checkpoint: dense MXFP8 through an SM80 Marlin W8A16 path, routed experts MXFP4 through W4A16; KV FP8 (1)
engine:
Custom SM80 vLLM (patched from public commit 79a7108d9) (1) · ExLlamaV3 1.5.4 (built for SM80) with a small OpenAI-compatible server (1) · Project Maya (Strata and llama.cpp lineage) (1) · vLLM (build not stated) (1) · vLLM nightly + PR #53899 (a5530b9) + the repo's patch scripts (1)
engine version:
None (2) · 0.1.dev1+ga5530b90c.d20260829 (1) · Flun/glm53-flash-cmp170hx-exl3 at 5933112 (1) · reports 0.0.0; patches/public-base-to-runtime.patch (109 files) (1)
cards:
8 GB (5)
card count:
2 (3) · 6 (1) · 8 (1)
topology:
engine-managed expert tiering across VRAM, host RAM and NVMe (1) · layer split (no tensor parallel), no P2P (1) · PP2 with PLE CPU offload (1) · PP8, TP1 (1) · TP6 (virtual-head padding), PP1 (1)
context:
None (1) · 1,048,576 (2) · 262,144 (1) · 392,960 (profile q8_384k) (1)
decode tps:
25 (1) · 60 (1) · 63 (1) · 75.4 (1) · 78.55 (1)
prefill tps:
None (5)
vram per card:
64 GB (1) · 64 GB, --gpu-memory-utilization 0.95 (1) · 64 GiB, --gpu-memory-utilization 0.97 (1) · 64 GiB; weights PP0 39.48 GiB, PP1 45.53 GiB; --gpu-memory-utilization 0.9 (1) · about 58 GiB of 64 GiB (116.6 GiB resident across both) (1)
power cap:
None (4) · 200 W (1)
driver:
None (5)
unlock build:
None (5)
status:
Proven (1) · Reported (4)
tested by:
1a2b3c0967 (1) · 68x0 (1) · pcsv. (1) · vektor_prime (1) · wgkbfcp (1)
tested on:
August 2026 (1) · October 2026 (4)
source:
https://discord.com/channels/1518634379604136178/1528826330454429706/1557523316120748073 (1) · https://github.com/Flun/glm53-flash-cmp170hx-exl3 (1) · https://github.com/mw00/project-maya#how-fast-is-it (1) · https://github.com/vektorprime/qwen38-flash-next-pp2 (1) · https://github.com/wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe (1)

Showing below up to 5 results in range #1 to #5.

View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)


View (previous 250 | next 250) (20 | 50 | 100 | 250 | 500)

Retrieved from "https://cmp.wiki/wiki/Special:Drilldown/Recipe"