DeepSeek V4.1 Flash on 2x 8 GB (Project Maya)
More actions
| Reported 2 x 8 GB | |
| Quant | Maya GGUF (Maya-S 340 GB or Maya-XS 277 GB; Engram tables stay on SSD) |
|---|---|
| Engine | Project Maya (Strata and llama.cpp lineage) |
| Topology | engine-managed expert tiering across VRAM, host RAM and NVMe |
| Decode | 63 tok/s |
| VRAM per card | 64 GB |
| Source | link |
| Confirmations | None yet. |
Most of DeepSeek-V4.1-Flash does not fit on two cards, so this setup keeps the hot experts and KV in VRAM and moves the
rest between host RAM and NVMe as needed. pcsv quotes the Project Maya README at about 63 tok/s for DeepSeek 4.1 Flash on
two 8 GB-variant cards. Strata, the engine Maya started from, gives 100+ tok/s on CMPs with default settings for models
that fit. No reporter, host or quant was named with the 63 tok/s figure.
Source: @pcsv., 2026-10-25
Source: @pcsv., 2026-10-25
Prerequisites
- Two CMP 170HX at 64 GB (Driver unlock, 8 GB card).
- A lot of host RAM and a fast NVMe: the engine pins all but about 6 GB of free host RAM for its expert tier; 32 GB is the minimum it accepts, and Maya-S is 340 GB on disk.
- Linux x86-64 with AVX2, CUDA 12.x, g++, Python 3.10+.
Source: repo github.com/mw00/project-maya by mw00 on GitHub, posted by @pcsv., 2026-10-25
Steps
Follow the Project Maya README with the DeepSeek setup (Maya-S, or Maya-XS to keep the Engram tables on SSD), then
./maya.sh --calibrate (10 to 15 minutes) and ./maya.sh --bench. Weights from the
peasantsmith Hugging Face page; it also loads LibertAI's EXL3 2.9 and 3.5 bpw packs
and unsloth GGUFs. See GLM 5.3 Flash on 2x 8 GB (Project Maya) for the engine's commands and API.
Source: repo github.com/mw00/project-maya by mw00 on GitHub, posted by @pcsv., 2026-10-25
Measured numbers
- About 63 tok/s decode on 2x CMP 8 GB (README figure). The README claims DeepSeek Maya-S matches the original's next token 92.6% of the time. Source: @pcsv., 2026-10-25
- 68x saw prefill of only 500 to 550 tok/s. pcsv: prefill depends heavily on PCIe and RAM, because experts and KV move dynamically when they do not fit on the card. Source: @68x0, 2026-10-25
Source: @pcsv., 2026-10-25 - For scale: Kimi K3 ran on a Strata fork (ro99/Strata) at 38.6 s per token with 64 GB of VRAM and 254 GB of system RAM. Source: @segmond, 2026-10-25
Plans and estimates
- Theory Nestaa51 planned to run ISTA-DASLab's DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF on his two cap-modded 8 GB cards with RAM offload (estimate: 20 to 30 GB offload plus KV, so 40 to 50 GB of host RAM after the cards fill; he has 128 GB DDR4 and a 990 Pro) and compare it with GLM 5.3 Flash EXL3. No result posted. Source: @nestaa51, 2026-10-24
- Theory Petri Krohn's estimate for one 64 GB card: it holds about 19% of the experts for an 80% cache hit rate; with 1,500 GB/s HBM and about 16 GB of active parameters the ceiling is about 100 tok/s at the native 4-bit quant without MTP, 150 to 200 with MTP, and the CPU needs about 75 GB/s of memory bandwidth to keep up. Replies: PCIe 4.0 x16 is only 31.5 GB/s per direction, but offload engines such as Strata keep the hottest experts on the GPU and touch system RAM only on a miss. Source: @petrikrohn_26497, 2026-10-21
Source: @wakeupmbrc1, 2026-10-21
Related
- GLM 5.3 Flash on 2x 8 GB (Project Maya), Engines (Strata and Project Maya), Benchmarks.
- With four cards everything fits in VRAM: DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake).
Sources
- mw00/project-maya (MIT), posted on Discord by pcsv on 2026-10-25.