GLM 5.3 Flash on 2x 8 GB (Project Maya)

GLM-5.3-Flash (321B in the Maya README)
Reported 2 x 8 GB
Quant	Maya-M GGUF (116 GB), every expert in VRAM; Maya-S (96.5 GB) and Maya-S24 (94.7 GB) also reported
Engine	Project Maya (Strata and llama.cpp lineage, EXL3 loading, DFlash2 drafter) v1.0.6 (this row)
Topology	engine-managed split with expert tiering across VRAM, RAM and NVMe
Decode	61.3 tok/s
Prefill	514 tok/s
VRAM per card	64 GB
Flags	`./maya.sh --bench (three greedy 256-token answers for decode, an 8K-token prompt for prefill)`
Tested	@ZackO2o
Source	link
Confirmations	None yet.

Project Maya is a self-tuning inference engine with a web chat and OpenAI- and Anthropic-compatible APIs. It runs GLM-5.3-Flash, DeepSeek-V4.1-Flash-1017 and MiniMax-M3.1-Flash by tiering MoE experts across VRAM, RAM and NVMe, so it also works when the model does not fit in the cards. It needs compute capability 7.0 or newer, so the CMP 170HX (sm_80) is supported. 68x names it, with the Flun forks, as the way to run GLM 5.3 Flash on two CMPs. Source: @pcsv., 2026-10-25
Source: @68x0, 2026-10-26

Prerequisites

NVIDIA compute capability 7.0+, one to 16 GPUs, 32 GB host RAM minimum, a fast NVMe (not an HDD), Linux x86-64 with AVX2, CUDA 12.x, g++, Python 3.10+.
The engine pins all but about 6 GB of free host RAM for its expert tier.
Unlocked cards help: with every expert in VRAM (Maya-M on two 64 GB cards) nothing streams from RAM. See Driver unlock.

Source: repo github.com/mw00/project-maya by mw00 on GitHub, posted by @pcsv., 2026-10-25

Steps

Clone mw00/project-maya and follow its README for the first build (20 to 40 minutes plus the model download) and the model setup id (glm-maya-s by default). The documented maya.sh commands after that:

./maya.sh --calibrate        # self-tuning, 10 to 15 minutes
./maya.sh --bench            # decode (three greedy 256-token answers) and prefill (an 8K-token prompt)
./maya.sh --report           # writes maya-report.txt to attach to an issue

Quants for GLM-5.3-Flash: Maya-S 96.5 GB (glm-maya-s, default), Maya-S24 94.7 GB (glm-maya-s24, for cards of 24 GB or less, decode +14%), Maya-M 116 GB, Maya-M-Derisked 116 GB (an experimental refusal-reducing weight edit), Maya-L 156.3 GB. Weights: peasantsmith/GLM-5.3-Flash-Maya-GGUF.
It also loads third-party packs such as turboderp's GLM EXL3. ./maya.sh --import-model FOLDER adopts files downloaded elsewhere.
API: http://127.0.0.1:8080/v1, Anthropic-style /v1/messages, up to 8 concurrent requests (--parallel). For remote access, ./maya.sh --setup --host 0.0.0.0 --api-key <secret>.

Source: repo github.com/mw00/project-maya by mw00 on GitHub, posted by @pcsv., 2026-10-25

Measured numbers

CMP 170HX rows from the README's user bench table (each is one user's report of one version, not a controlled test):

Cards	Host	Quant and context	Decode tok/s	Prefill tok/s	Reporter
2x 64 GB, PCIe Gen2 x4 each	2x Xeon E5-2690 v4, 91 GB RAM	Maya-M, every expert in VRAM	61.3	514	ZackO2o, v1.0.6, issue #22
2x listed as "40 GB", PCIe Gen2 x16 each	Xeon w3-2423, 128 GB DDR5	Maya-S, 128K	54.3	831	nvsofts, v1.1.15, issue #263
1x 64 GB, PCIe Gen2 x16	Core i7-9700K, 32 GB RAM	Maya-S24, 32K	20.6	797	Kanorin-chan, v1.1.8, issue #281

The "40 GB" row is quoted as written; how those cards were unlocked is not stated. Source: repo github.com/mw00/project-maya by mw00 on GitHub, posted by @pcsv., 2026-10-25

Quality

The README claims the GLM Maya quants keep 97.7 to 99.2% of the FP8 model's zero-shot accuracy and pick its next token 83 to 90% of the time (Maya-L 99.2% / 90%).
PJ asked how Maya-L (3 to 4-bit experts IQ3_S / IQ4_XS, Q5_K in the most sensitive layers) compares; Gnasherrr's view: Q5_K probably similar, 3-bit probably worse than W4A16, and most people run W4A16 on four cards. Source: @zingy90210, 2026-10-27
Source: @gnasherrr, 2026-10-27

Known issues

Prefill depends heavily on PCIe and host RAM, because experts and KV cache move dynamically when they do not fit on the card. Source: @pcsv., 2026-10-25
Parts of the Maya fork were merged back into Strata mainline; pcsv says without proper attribution. Source: @pcsv., 2026-10-25
The optional DFlash2 drafter (by Anbeeld) is CC BY-NC-ND 4.0, non-commercial.

Sources

Repo: mw00/project-maya (MIT; by Peasant Smith, co-maintained by needmorevram). Posted on Discord by pcsv on 2026-10-25.