View source for DeepSeek V4.1 Flash on 2x 8 GB (Project Maya)

{{Recipe
|model=DeepSeek-V4.1-Flash-1017 (748B in the Maya README)
|quant=Maya GGUF (Maya-S 340 GB or Maya-XS 277 GB; Engram tables stay on SSD)
|engine=Project Maya (Strata and llama.cpp lineage)
|cards=8 GB
|card_count=2
|topology=engine-managed expert tiering across VRAM, host RAM and NVMe
|decode_tps=63
|vram_per_card=64 GB
|status=Reported
|tested_by=pcsv.
|tested_on=2026-10-25
|source=https://github.com/mw00/project-maya#how-fast-is-it
}}
Most of DeepSeek-V4.1-Flash does not fit on two cards, so this setup keeps the hot experts and KV in VRAM and moves the
rest between host RAM and NVMe as needed. pcsv quotes the Project Maya README at about 63 tok/s for DeepSeek 4.1 Flash on
two 8 GB-variant cards. Strata, the engine Maya started from, gives 100+ tok/s on CMPs with default settings for models
that fit. pcsv posted the figure; no host or quant was named with it.
{{Source|handle=pcsv.|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563924151662678087}}<br />
{{Source|handle=pcsv.|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563917658334695467}}

== Prerequisites ==
* Two CMP 170HX at 64 GB ([[Driver unlock]], [[8 GB card]]).
* A lot of host RAM and a fast NVMe: the engine pins all but about 6 GB of free host RAM for its expert tier; 32 GB is the minimum it accepts, and Maya-S is 340 GB on disk.
* Linux x86-64 with AVX2, CUDA 12.x, g++, Python 3.10+.
{{Source|handle=pcsv.|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563923997970796645|repo=https://github.com/mw00/project-maya}}

== Steps ==
Follow the Project Maya README with the DeepSeek setup (Maya-S, or Maya-XS to keep the Engram tables on SSD), then
<code><nowiki>./maya.sh --calibrate</nowiki></code> (10 to 15 minutes) and <code><nowiki>./maya.sh --bench</nowiki></code>. Weights from the
[https://huggingface.co/peasantsmith peasantsmith] Hugging Face page; it also loads LibertAI's EXL3 2.9 and 3.5 bpw packs
and unsloth GGUFs. See [[GLM 5.3 Flash on 2x 8 GB (Project Maya)]] for the engine's commands and API.
{{Source|handle=pcsv.|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563923997970796645|repo=https://github.com/mw00/project-maya}}

== Measured numbers ==
* About 63 tok/s decode on 2x CMP 8 GB (README figure). The README claims DeepSeek Maya-S matches the original's next token 92.6% of the time. {{Source|handle=pcsv.|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563924151662678087}}
* 68x saw prefill of only 500 to 550 tok/s. pcsv: prefill depends heavily on PCIe and RAM, because experts and KV move dynamically when they do not fit on the card. {{Source|handle=68x0|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563951775135043595}}<br />{{Source|handle=pcsv.|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563952834649526424}}
* For scale: Kimi K3 ran on a Strata fork (ro99/Strata) at 38.6 s per token with 64 GB of VRAM and 254 GB of system RAM. {{Source|handle=segmond|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563921194116186174}}

== Plans and estimates ==
* {{Status|Theory}} Nestaa51 planned to run ISTA-DASLab's [https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF] on his two cap-modded 8 GB cards with RAM offload (estimate: 20 to 30 GB offload plus KV, so 40 to 50 GB of host RAM after the cards fill; he has 128 GB DDR4 and a 990 Pro) and compare it with GLM 5.3 Flash EXL3. No result posted. {{Source|handle=nestaa51|date=2026-10-24|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563530143144873986}}
* {{Status|Theory}} Petri Krohn's estimate for one 64 GB card: it holds about 19% of the experts for an 80% cache hit rate; with 1,500 GB/s HBM and about 16 GB of active parameters the ceiling is about 100 tok/s at the native 4-bit quant without MTP, 150 to 200 with MTP, and the CPU needs about 75 GB/s of memory bandwidth to keep up. Replies: @shudza asked whether it is PCIe bound first, and @nestaa51 noted PCIe 4.0 x16 is only 31.5 GB/s per direction; @wakeupmbrc1: offload engines such as Strata do not recount all the experts every token, they keep the hottest experts on the GPU. @sojun80: "strata moves them in and out of memory on to the gpu. tho, i suspect, for 170hx's we'd do better to have them all in memory all of the time" (which memory he means is not stated). {{Source|handle=petrikrohn_26497|date=2026-10-21|link=https://discord.com/channels/1518634379604136178/1518695328008638516/1562496635349630981}}<br />{{Source|handle=shudza|date=2026-10-21|link=https://discord.com/channels/1518634379604136178/1518695328008638516/1562523139869507595}}<br />{{Source|handle=nestaa51|date=2026-10-21|link=https://discord.com/channels/1518634379604136178/1518695328008638516/1562524603748712480}}<br />{{Source|handle=wakeupmbrc1|date=2026-10-21|link=https://discord.com/channels/1518634379604136178/1518695328008638516/1562526221303283853}}<br />{{Source|handle=wakeupmbrc1|date=2026-10-21|link=https://discord.com/channels/1518634379604136178/1518695328008638516/1562526312260702320}}<br />{{Source|handle=sojun80|date=2026-10-21|link=https://discord.com/channels/1518634379604136178/1518695328008638516/1562532238183366657}}

== Related ==
* [[GLM 5.3 Flash on 2x 8 GB (Project Maya)]], [[Engines]] (Strata and Project Maya), [[Benchmarks]].
* With four cards everything fits in VRAM: [[DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake)]].

== Sources ==
* [https://github.com/mw00/project-maya#how-fast-is-it mw00/project-maya] (MIT), posted on Discord by pcsv on 2026-10-25.

{{ImportFooter}}
[[Category:8 GB]]