GLM 5.3 Flash on 3x 8 GB
More actions
| Reported 3 x 8 GB | |
| Quant | W4A16, about 4.3 bpw, 163 GiB on disk |
|---|---|
| Engine | Morrowmake/vllm-cmp170hx (via the glm53-flash-cmp170hx-recipe) not stated |
| Topology | PP3 |
| Context | about 1,000,000 (8-bit KV cache) |
| VRAM per card | 64 GiB |
| Flags | Morrowmake's recipe with pipeline parallel 3; KV cache quantized to 8 bits |
| Tested | @shirogeek on 2026-10-19 |
| Source | Discord message |
| Confirmations | None yet. |
GLM-5.3-Flash on three cards, leaving the fourth free for image generation, mining or rental. Shirogeek runs a W4A16
quant of about 4.3 bpw (163 GiB) on three 64 GB cards. His own recipe converged with Morrowmake's, so his instruction
is: use Morrowmake's recipe with PP3. He quantizes the KV cache to 8 bits and gets a context window of about 1M tokens.
No tok/s figure was posted.
Source: @shirogeek, 2026-10-19
Source: @shirogeek, 2026-10-19
Prerequisites
Same as GLM 5.3 Flash on 4x 8 GB, with three cards instead of four: each card exposes 64 GiB (Driver unlock, 8 GB card), driver 580 or newer, Docker with the NVIDIA Container Toolkit. PP3 passes only activations between cards, so x4 links are workable.
Steps
Theory The exact 3-card configuration has not been published. Shirogeek offered to share his config after two weeks away from his rig (ask him in early November 2026). Until then, the path he describes is:
- Set up GLM 5.3 Flash on 4x 8 GB as written.
- Switch it to pipeline parallel over three cards (the recipe's
LAYOUT,TPandPPkeys; three cards is not one of the recipe's validated layouts). - Quantize the KV cache to 8 bits to reach about 1M tokens of context. The recipe ships a full-precision KV cache, so this is a change from its defaults.
Source: @shirogeek, 2026-10-19
Known issues
- The 163 GiB checkpoint is not the recipe's default (canada-quant/GLM-5.3-Flash-W4A16-MTP is about 178 GiB). Which 4.3 bpw W4A16 checkpoint he used was not stated.
- No speed numbers yet. Expect PP3 to behave like the recipe's PP4: strong prefill, slower single-stream decode than TP.
Other three-card options
- ExLlamaV3: Wingrime scaled Flun's two-card EXL3 repo to three cards with an AI agent and gets about 60 tok/s in TP mode at full context, but says the engine is suboptimal. 68x gets the same 60 tok/s on two cards without DFlash, so the third card buys context, not speed. See GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split). Source: @wingrimez_40467, 2026-10-20
Source: @68x0, 2026-10-20 - Why people ask: cibernox wants GLM Flash on three cards at about 3.5 bpw so the fourth can run image generation; Finishtilt14 wants it on one or two cards so the rest can mine or be rented. Source: @cibernox, 2026-10-13
Source: @finishtilt14, 2026-10-13 - Miséricorde's view: GLM needs four cards to run properly in integer weights; on two or three cards Qwen3.8 Flash Next is the better pick. See Recipes. Source: @wakeupmbrc1, 2026-10-13
Sources
- Shirogeek's posts in the testing channel, 2026-10-19.
- Base recipe: Morrowmake/glm53-flash-cmp170hx-recipe.
- Related: Engines, Quantization on the 170HX, Benchmarks.