GLM 5.3 Flash on 2x 8 GB (AlexBrandes vLLM EXL3 PP2)

GLM-5.3-Flash (320B MoE, 18B active)
Reported 2 x 8 GB
Quant	EXL3 2.6 bpw (turboderp); dense weights requantised from the original model to INT8 instead of EXL3
Engine	vLLM with EXL3 support (the author's fork)
Topology	PP2, no P2P
Links	PCIe Gen2 x4, no P2P
Context	225,280 tokens (README)
Decode, 1 user	70 tok/s
Prefill	2500 tok/s
Measured as	Author's Discord figures (decode a little over 70 tok/s, prefill 2,500); the README lists 76.0-101.7 tok/s for one request by content type
VRAM per card	64 GB (unlocked 8 GB cards, 128 GB across both)
Power cap	170 W per card (README measurements)
Tested	@alexbrandes on 2026-11-11
Source	link
Confirmations	None yet.

GLM-5.3-Flash on two unmodified 64 GB cards: no x16 capacitor mod and no P2P. AlexBrandes runs turboderp's 2.6 bpw EXL3 quant in vLLM as a two-stage pipeline (PP2). The dense weights are requantised from the original model to INT8 instead of EXL3; he says that speeds up decode by about 25% and should give higher quality. His post gives decode a little over 70 tok/s and prefill 2,500 tok/s. This is a different engine from the other two-card EXL3 builds, which run ExLlamaV3 itself (GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 TP2) and GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split)). Source: @alexbrandes, 2026-11-11
Source: repo github.com/AlexBrandes/cmp170hx-glm53-flash-exl3 by AlexBrandes on GitHub

Prerequisites

Two CMP 170HX 8 GB cards that each expose 64 GiB (Driver unlock, 8 GB card). The repository does not cover the unlock.
No x16 mod and no P2P: the README says Gen2 x4 is enough for this PP2 layout.
128 GB of VRAM in total. At 2.6 bpw the model leaves room for the KV cache; a 3.0 bpw quant does not quite fit.

Source: @alexbrandes, 2026-11-13
Source: repo github.com/AlexBrandes/cmp170hx-glm53-flash-exl3 by AlexBrandes on GitHub

Steps

The repository README has the setup, the model download and the launch commands; they are not reproduced here. Use AlexBrandes/cmp170hx-glm53-flash-exl3 as published. Source: repo github.com/AlexBrandes/cmp170hx-glm53-flash-exl3 by AlexBrandes on GitHub

Measured numbers

The README figures were measured by the author on 2026-11-11 at 170 W per card, PP2, no P2P.

Test	Result
Discord post, decode	a little over 70 tok/s
Discord post, prefill	2,500 tok/s
README, one request	76.0-101.7 tok/s, depending on content type
README, four requests	up to 222.9 tok/s
README, prefill from 4k to 128k tokens	about 2,420-2,541 tok/s
README, context window	225,280 tokens

Decode as reported: "a little over 70" in the post, 76.0-101.7 in the README; the infobox uses the post's figure. fallengod9962 says the recipe works well for him; he posted no numbers, so this is not a confirmation. Source: @alexbrandes, 2026-11-11
Source: @fallengod9962, 2026-11-13
Source: repo github.com/AlexBrandes/cmp170hx-glm53-flash-exl3 by AlexBrandes on GitHub

Notes

Other EXL3 quants. Asked for an uncensored model to use with it, AlexBrandes pointed to MikeRoz/GLM-5.3-Flash-Uncensored-2.51bpw-h6-exl3 as something that should work; he had not tried it. Any EXL3 quant has to leave room for the KV cache on 128 GB. Source: @alexbrandes, 2026-11-13
Why INT8 dense weights. The author says the EXL quant "can be a bit slow by default on CMP"; keeping the dense layers in INT8 is one of the tricks he found to speed it up. Why EXL3 decode is slow on these cards: Quantization on the 170HX. Source: @alexbrandes, 2026-11-11

Known issues

One author and one machine; the numbers are the author's own. A second owner reports that it works, with no numbers.
The quant is 2.6 bpw, below the 3.05 bpw of the ExLlamaV3 two-card builds. No quality comparison was posted.

Source: @fallengod9962, 2026-11-13

Sources

Repo: AlexBrandes/cmp170hx-glm53-flash-exl3. Source: repo github.com/AlexBrandes/cmp170hx-glm53-flash-exl3 by AlexBrandes on GitHub
Discord: AlexBrandes' announcement 2026-11-11 and his reply on other quants 2026-11-13; fallengod9962's report 2026-11-13.