Running GLM 5.3 Flash locally: hardware and measured speeds

Short answer. GLM-5.3-Flash (320B MoE, 18B active) runs entirely in VRAM on 8 GB cards unlocked to 64 GB. The community's standard build is four cards with W4A16 weights on Morrowmake's vLLM recipe: 442 tok/s for one user on code, 835 tok/s across eight users and 3,161 tok/s cold prefill at 180 W per card, with a 262,144-token context. Two cards run it at about 3 bpw (EXL3) at 60 to 108 tok/s. Eight cards run two four-card instances, or one instance with a 1,048,576-token context. Every row below links to the recipe page with the exact commands.

Speeds are from each recipe's own benchmark, usually one user on code at temperature 0 with a speculative drafter on. Real agent work is slower; see Known problems and Benchmarks.

Hardware it needs

Cards. Four unlocked 8 GB cards (256 GB) hold the W4A16 checkpoint (about 178 GiB) with room for the KV cache. Two cards hold it at EXL3 3.05 bpw (116.6 GiB resident across both). Cards must show 64 GiB each: see Driver unlock.
Links. Tensor parallel on four cards (TP4) needs PCIe Gen2 x16 on every card, which means the PCIe x16 capacitor mod. Cards on stock x4 links use pipeline parallel (PP4) on the same recipe, or one of the PP recipes below. P2P is optional on the four-card recipe and required for the two-card TP2 build.
Disk. About 180 GiB for the four-card recipe's two checkpoints (about 178 GiB model, 2.2 GiB drafter), plus the engine image and compile caches.
Power. The published four-card numbers are at 180 W per card. One owner sees about 104 W per card while generating under that cap.

Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06
Source: repo github.com/Flun/glm53-flash-cmp170hx-exl3 by Flun on GitHub, posted by @wingrimez_40467, 2026-10-20
Source: @peter643, 2026-10-20

Measured configurations

One row per recipe page. "One user" is the recipe's headline single-stream decode figure; "Many users" is the total across concurrent requests. Power caps, link widths and benchmark protocols differ between rows, so compare rows only after reading the recipe page. The status badge is the one each recipe page carries: Proven means a second person reproduced it.

Cards	Recipe	Weights	Layout	One user (tok/s)	Many users (tok/s)	Prefill (tok/s)	Context	Power cap	Status	Measured by
4	GLM 5.3 Flash on 4x 8 GB	W4A16 (INT4), about 178 GiB	TP4 on x16 links with P2P (PP4 for x4 links); DFlash2 drafter	442.1	835 at 8 users	3,161	262,144	180 W	Proven	voidgiraffe
4	GLM 5.3 Flash on 4x 8 GB (kaka86mm EXL3 + Marlin sidecar + DFlash2)	EXL3 4 bpw (164 GB) plus a Marlin INT4 sidecar (151 GB)	PP4 on stock x4 links, no P2P; DFlash2 drafter	96	199 at 4 users	2,077 (131k tokens)	524,288	200 W	Reported	dkpoulsen
2	GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 TP2)	EXL3 3.05 bpw	TP2 on x16 links with P2P; DFlash2 drafter	107.67	67.25 at 2 users	1,533 (32K tokens)	392,960 per request		Reported	nestaa51
2	GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split)	EXL3 3.05 bpw	layer split, no P2P, no x16 mod	60			392,960		Proven	68x0
2	GLM 5.3 Flash on 2x 8 GB (Project Maya)	Maya-M GGUF (116 GB)	engine-managed split on x4 links; DFlash2 drafter	61.3		514 (8K tokens)			Reported	ZackO2o
2	GLM 5.3 Flash on 2x 8 GB (AlexBrandes vLLM EXL3 PP2)	EXL3 2.6 bpw (turboderp) with INT8 dense weights	PP2 in vLLM on stock x4 links, no P2P	70	up to 222.9 at 4 users (README)	2,500	225,280	170 W	Reported	alexbrandes
3	GLM 5.3 Flash on 3x 8 GB	W4A16, about 4.3 bpw (163 GiB)	PP3	not posted			about 1,000,000 (8-bit KV cache)		Reported	shirogeek
8	GLM 5.3 Flash on 8x 8 GB (promisezackr PP8 NVFP4 DFlash2)	NVFP4 (181 GiB), run through INT4 kernels	PP8 on stock x4 links, no P2P; DFlash2 drafter	123	276 at 16 users		1,048,576		Reported	promisezackr

Sources for the rows (each recipe page carries its full credits):
Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06
Source: repo github.com/kaka86mm/glm53-flash-170hx by kaka86mm on GitHub, posted by @dkpoulsen, 2026-09-19
Source: repo github.com/alexvnesta/glm53-flash-cmp170hx-tp by alexvnesta on GitHub, posted by @nestaa51, 2026-10-12
Source: repo github.com/Flun/glm53-flash-cmp170hx-exl3 by Flun on GitHub, posted by @wingrimez_40467, 2026-10-20
Source: @pcsv., 2026-10-25
Source: repo github.com/mw00/project-maya by mw00 on GitHub
Source: @shirogeek, 2026-10-19
Source: repo github.com/AlexBrandes/cmp170hx-glm53-flash-exl3 by AlexBrandes on GitHub, posted by @alexbrandes, 2026-11-11
Source: repo github.com/promisezackr/glm53-flash-170hx-pp8 by promisezackr on GitHub, posted by @zanooda, 2026-08-31

Four cards in detail

Author's release 1.8.0 figures at 180 W per card, x16 links, P2P verified, 400-token answers at temperature 0, as structured / code / prose tok/s:

Layout	1 user	8 users	Cold prefill	KV pool (tokens)
TP4	482.6 / 442.1 / 232.1	943.7 / 835.0 / 667.2	3,161	1,203,765
PP4	232.4 / 209.9 / 100.1	664.1 / 608.8 / 469.2	7,419	2,088,960

TP4 is faster for one user; PP4 reads prompts faster, holds more context and works on x4 links.

Reported 8 GB
Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06

What other owners measured on the same recipe:

cibernox (x16, P2P on, TP4): over 350 tok/s in short bursts and 270+ tok/s on real workflows. Source: @cibernox, 2026-10-13
turbine (TP4): 317 tok/s at batch 1 on code, 164 on prose. Source: @turbine, 2026-10-27
Mountains (PP4 on Gen2 x4 links): 90 to 140 tok/s decode with a 1M context window, prefill around 3k tok/s. Source: @mountains1605, 2026-10-04
PixelML (four cards behind two PLX switches, release 1.6.0, P2P off, 180 W): 396.0 tok/s structured for one user and 798.6 at eight users. Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @c133, 2026-10-20
The lab serves the recipe with a cut-through ring all-reduce in place of the host-staged one: 18 to 20% more tok/s at 8 users than the release image on the same cards in the same session, cold prefill unchanged. Build steps are on the recipe page. Source: @darkmaniac7, 2026-11-10 (lab notes)
Source: @darkmaniac7, 2026-11-11 (lab notes)

Eight cards

Two instances of the four-card recipe on eight cards give about 1,600 tok/s aggregate (two PP4 groups, or two TP4 groups if the cards have the x16 mod), at a 180 to 200 W power limit. Source: @impactic, 2026-10-20

One instance across all eight cards (PP8, NVFP4 weights) trades speed for context: 123 tok/s for one stream on code, a 1,048,576-token context and a 6.62M-token KV pool, on stock x4 links. Those are the patch author's own readings; nobody has posted a matching eight-card run. Source: repo github.com/promisezackr/glm53-flash-170hx-pp8 by promisezackr on GitHub, posted by @zanooda, 2026-08-31

The four-card recipe stretched into one instance on eight cards is slower than two four-card instances. On the lab's eight-card server at 250 W, TP8 gave 56 to 105 tok/s for one user and 440 at 8 users with 554 tok/s prefill, and TP4 x PP2 gave 38 to 145 for one user and paid off only at 32 users (492 aggregate) and in prefill (1,856), because the all-reduce over Gen2 and the hop between CPU sockets dominate. Two TP4 instances, one per socket, is the better use of eight cards. See GLM 5.3 Flash on 4x 8 GB#Layouts. Reported Source: @darkmaniac7, 2026-10-27 (lab notes)

Known problems

The model always reasons. Set max_tokens for reasoning plus answer, or content comes back empty with finish_reason: "length". enable_thinking: false is ignored; use reasoning_effort (low, high, max).
Older four-card releases. Release 1.7 had a TP4 lost-context and endless-repetition bug, fixed in 1.7.2 along with GPU memory growth over hours. Use the current release.
Non-commercial drafter. The default DFlash2 drafter is CC BY-NC-ND 4.0, so the published configuration is non-commercial, and converted copies must not be shared.

Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06

Benchmark numbers against real use. Contested The release figures use temperature 0 and synthetic prompts, which let the drafter hit unusually often. cibernox, who matched the published numbers, expects real coding to be slower; Morrowmake measures about 200 tok/s in his normal sessions. Source: @cibernox, 2026-10-02
Source: @voidgiraffe, 2026-10-03
Two-card EXL3 builds. The layer-split build serves one request at a time. On the TP2 build a large new prompt slows an active session to about a third of its speed until its prefill finishes. Source: repo github.com/Flun/glm53-flash-cmp170hx-exl3 by Flun on GitHub, posted by @wingrimez_40467, 2026-10-20
Source: repo github.com/alexvnesta/glm53-flash-cmp170hx-tp by alexvnesta on GitHub, posted by @nestaa51, 2026-10-12
EXL3 4 bpw build on four cards. Its CPU KV tier was withdrawn on 2026-09-19 after corrupted output; turn it off by hand, because the repository's launch script still enables it. Source: repo github.com/kaka86mm/glm53-flash-170hx by kaka86mm on GitHub, posted by @dkpoulsen, 2026-09-19
PP8 on eight cards. One owner could not get past 20 tok/s and kept hitting Xid errors; another fixed the same Xid errors by updating the unlocker to version 0.3. See Xid errors. Source: @1a2b3c0967, 2026-09-01
Source: @zanooda, 2026-09-01

More: Known bad combinations, Instability under load, vLLM on sm_80: known bugs and patches.

Common questions

How much VRAM does GLM 5.3 Flash need?

The W4A16 checkpoint the four-card recipe uses is about 178 GiB, so it needs four CMP 170HX 8 GB cards unlocked to 64 GB each (256 GB). At EXL3 3.05 bpw it is 116.6 GiB resident and fits on two unlocked cards. See #Hardware it needs.

How many CMP 170HX cards do I need for GLM 5.3 Flash?

Four is the standard build (W4A16, 442 tok/s for one user on code). Two cards work with a 3.05 bpw EXL3 quant at 60 to 108 tok/s. Three cards run it in pipeline parallel with a W4A16 quant of about 4.3 bpw, with no speed posted. See #Measured configurations.

How fast is GLM 5.3 Flash on four CMP 170HX cards?

The recipe author's figures at 180 W per card are 482.6 / 442.1 / 232.1 tok/s for one user on structured, code and prose answers, 835 tok/s on code across eight users, and 3,161 tok/s cold prefill (TP4, x16 links, P2P). Other owners report 270 to 350 tok/s on the same recipe in daily use.

Can GLM 5.3 Flash run on two CMP 170HX cards?

Yes, at about 3 bpw. The ExLlamaV3 layer-split build needs no P2P and no x16 mod and decodes at about 60 tok/s; the TP2 build needs x16 links and P2P and reaches about 108 tok/s for one request. Both give a 392,960-token context. A third build runs a 2.6 bpw EXL3 quant in vLLM as PP2 on stock x4 links without P2P, at a little over 70 tok/s with a 225,280-token window.

Does GLM 5.3 Flash need the x16 mod or P2P?

Only for tensor parallel. TP4 on four cards needs Gen2 x16 on every card; on stock x4 links run the same recipe as PP4 (about 210 tok/s for one user on code in the author's table, and faster prefill). P2P is optional on the four-card recipe.

What context length does GLM 5.3 Flash get on CMP 170HX cards?

262,144 tokens per request on the four-card recipe, with a KV pool of about 1.2M tokens at TP4 and about 2.1M at PP4. The eight-card PP8 build and the three-card PP3 setup (8-bit KV cache) reach about 1M tokens per request.

Can I use the GLM 5.3 Flash recipe commercially?

Not as published. The default DFlash2 drafter is licensed CC BY-NC-ND 4.0, which is non-commercial. Check each recipe page for its own licence notes.

Recipe pages for this model are in the table above; Recipe finder filters all recipes by cards, model and engine.
The full model: Running GLM 5.3 locally: hardware and measured speeds.
Other model pages: Running DeepSeek V4.1 Flash locally: hardware and measured speeds, Running Qwen3.8 Flash Next locally: hardware and measured speeds.
Recipes, Benchmarks, Engines, Quantization on the 170HX, Speculative decoding on the 170HX, Parallelism over the PCIe link.