GLM 5.3 Flash on 4x 8 GB

GLM-5.3-Flash (320B MoE, 18B active)
Proven 4 x 8 GB
Quant	W4A16 (INT4 weights, group size 128, FP16 activations): canada-quant/GLM-5.3-Flash-W4A16-MTP
Engine	Morrowmake/vllm-cmp170hx (vLLM fork, container image) recipe 1.8.0, engine 819146023f
Topology	TP4 (default) or PP4
Context	262,144
Decode	442.1 tok/s
Prefill	3161 tok/s
VRAM per card	64 GiB exposed, --gpu-memory-utilization 0.95
Power cap	180 W
Driver	NVIDIA 580 or newer; P2P needs the Morrowmake/cmpunlocker build
Unlock build	cmpunlocker (64 GiB, 74 SMs); Morrowmake/cmpunlocker for P2P
Flags	`LAYOUT=tp4 or LAYOUT=pp4, P2P=auto, --max-num-seqs 8, --gpu-memory-utilization 0.95, --max-num-batched-tokens 3463 (TP4) or 2312 (PP4)`
Tested	@voidgiraffe on 2026-10-16
Source	link
Confirmations	Reproduced by @turbine (8 GB), 2026-10-27 · details Reproduced by @darkmaniac7 (8 GB), 2026-10-19 · details Reproduced by @cibernox (8 GB), 2026-10-13 · details Reproduced by @mountains1605 (8 GB), 2026-10-08 · details Reproduced by @ikyo (8 GB), 2026-10-06 · details Reproduced by @1a2b3c0967 (8 GB), 2026-10-04 · details

The community's reference build for four cards: GLM-5.3-Flash in W4A16 on Morrowmake's Ampere vLLM fork, with a DFlash2 speculative drafter, tool calls, reasoning, images and video, behind an OpenAI-compatible API. Most 4-card owners run W4A16 on this recipe, and Impactic says nobody really cares to upgrade past it. Infobox numbers are the author's TP4 release figures for one user on code. Source: @gnasherrr, 2026-10-27
Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06
Source: @impactic, 2026-10-24

Prerequisites

Four CMP 170HX that each expose 64 GiB (nvidia-smi shows 65,536 MiB). The preflight stops below 61,440 MiB. See Driver unlock and 8 GB card.
PCIe Gen2 x16 links for TP4 (the x16 capacitor modification). Cards stuck at x4 should use PP4. See Unlocks.
NVIDIA driver 580 or newer, Docker with the NVIDIA Container Toolkit. docker run --rm --gpus all nvidia/cuda:13.3.1-base-ubuntu24.04 nvidia-smi must list all four cards.
Optional: a P2P-capable driver build (Morrowmake/cmpunlocker: mainline cmpunlocker plus the maintainer P2P patch and the TRAP31 PLM patch, ForceP2P=0x11). Without it the server runs with P2P off.
About 180 GiB (193 GB) of disk for the two checkpoints (about 178 GiB model, 2.2 GiB drafter), plus the engine image and compile caches.
git, curl, flock, setsid, python3, jq.

Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06

Steps

git clone https://github.com/Morrowmake/glm53-flash-cmp170hx-recipe.git
cd glm53-flash-cmp170hx-recipe
./start.sh          # pull the engine image by digest, fetch the model, start, wait for /health
./start.sh smoke    # one chat request and one tool call

Pick the layout with one variable, either in .env or as a prefix: LAYOUT=pp4 ./start.sh restart. ./start.sh checks link width and suggests PP4 if a card is narrower than x16. Docker Compose is the alternative (docker compose up); use one or the other. Other subcommands: status, logs, stop, restart, update; DRY=1 ./start.sh prints the launch without running it.

Test request:

curl http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model": "glm-5.3-flash", "messages": [{"role": "user", "content": "Write a haiku about PCIe."}], "max_tokens": 4096}'

The API listens on 127.0.0.1:8000 with no key unless HOST and API_KEY are set in .env. The first boot compiles kernels into ./cache and is slow; "No available shared memory broadcast block found in 60 seconds" lines during that compile are harmless. Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06

What the release pins

Engine: Morrowmake/vllm-cmp170hx branch ampere at 819146023facb31c04f1f8512e52464835a10d40 (tag glm53-recipe-1.8.0), on upstream vLLM 4e0681ed56. Image ghcr.io/morrowmake/vllm-cmp170hx:engine-819146023f, digest sha256:18a2f26207568128e1d792ce923e86a74e40def1e43e492db83132cab8cdbe14.
Model: canada-quant/GLM-5.3-Flash-W4A16-MTP at revision 5681f7c2 (base zai-org/GLM-5.3-Flash).
Drafter: incoai/GLM-5.3-Flash-DFlash2 at revision bf582e4e. It drafts up to 7 tokens for one user, 5 at two, 3 under load.
KV cache full precision, no FP8 anywhere, no CPU or disk offload. Defaults: 262,144 context, --max-num-seqs 8, --gpu-memory-utilization 0.95, prefill chunks of 3,456 tokens (TP4) or 2,304 (PP4), fair prefill in 384-token slices, prefix caching on, glm47 reasoning and tool parsers.
Every performance feature is a one-variable kill switch set by serve.sh. Keep PREFILL_CAP=0. VLLM_CUSTOM_ALLREDUCE_ALGO=2stage is set because the built-in crossover is tuned for NVLink. 1.8.0 renamed every VLLM_GLM5_* switch; old names are ignored with a warning.

Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06

Pin the release tag, not the branch: the fork's ampere branch was rebased onto new upstream on 2026-10-26, more than 1,000 commits past 1.8.0. Source: @darkmaniac7, 2026-10-26 (lab notes)

P2P

With P2P=auto (default) the recipe content-checks every ordered GPU pair at start (cudaMemcpyPeer, SM-kernel peer reads and writes, a separate-process CUDA IPC write; 128 KiB and 512 KiB with one byte either side, plus 1, 8 and 32 MiB). A pass sets VLLM_CUSTOM_ALLREDUCE_PCIE_P2P=1 and NCCL_P2P_LEVEL=SYS; a fail sets NCCL_P2P_DISABLE=1 and TP uses a host-staged all-reduce. nvidia-smi topo saying OK is not a data-integrity test. See Unlocks for the P2P driver. Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06

Measured numbers

Author's release 1.8.0, measured 2026-10-16 at 74 SMs and 180 W per card, PCIe x16, P2P verified, MiaAI-Lab protocol (400 tokens, temperature 0). Tok/s as structured / code / prose.

Layout	1 user	2 users	4 users	8 users	Cold prefill	KV pool
TP4	482.6 / 442.1 / 232.1	606.5 / 460.3 / 314.7	717.7 / 618.7 / 501.4	943.7 / 835.0 / 667.2	3,161	1,203,765
PP4	232.4 / 209.9 / 100.1			664.1 / 608.8 / 469.2	7,419	2,088,960

Quality (TP4): HumanEval 151/164, GSM8K 1,286/1,319. Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06

Other people's numbers on this recipe

cibernox (daily driver, x16, P2P on, TP4): over 350 tok/s in short bursts, 270+ tok/s on real workflows. Source: @cibernox, 2026-10-13 About 800 tok/s aggregate across agents. Source: @cibernox, 2026-10-19
turbine (TP4): 317 tok/s at batch 1 on code, 164 on prose. Source: @turbine, 2026-10-27
Mountains (PP4 on PCIe Gen2 x4 links): 90 to 140 tok/s decode with a 1M context window, prefill around 3k tok/s. Source: @mountains1605, 2026-10-04 340 tok/s total at 8 concurrent on real agent work. Source: @mountains1605, 2026-10-08
ikyo (first 4-card run, an early release Morrowmake said had bugs; 768-token answers, reasoning counted): concurrency 1 161.8 tok/s per user; 2: 124.3 per user, 231.0 aggregate; 4: 92.6 / 307.5; 8: 51.7 / 296.1 with 4.55 s median TTFT. Single user by workload: structured 182.1, code 161.8, prose 141.2. Source: @ikyo, 2026-10-06
1a2b3c0967 (release 1.7, PP4): concurrency 3, 1M context, KV pool about 3M tokens, still over 75 tok/s on prose. Source: @1a2b3c0967, 2026-10-04
peter643 (4x 64 GB capped at 180 W, P2P on; recipe not named): 200 to 400 tok/s at single concurrency, about 104 W per card while generating, just over 400 W for the four cards. Source: @peter643, 2026-10-20
PixelML's Club CMP 170HX (four cards behind two PLX switches on a Broadwell host, release 1.6.0): 396.0 tok/s structured for one user and 798.6 at eight users at 180 W with P2P off, matching Morrowmake's own EPYC P2P-off figures (394.0 / 797.9); 418.0 / 816.5 after equalising HBM clocks, at 150 W. Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @c133, 2026-10-20
Ivan (8-card box, two TP4 instances of this recipe, abliterated W4A16 + MTP checkpoint in place of canada-quant's, 1,024-token answers, effort low, NCCL only): see the layout table below. Source: @darkmaniac7, 2026-10-28 (lab notes)

Layouts

The recipe ships TP4 and PP4. Measured trade-offs from the Discord:

TP4 over PCIe Gen2 needs x16 links. On x4 links decode is usable but prefill is not, and PP4 feels snappier in conversation because prefill drives that feeling. Source: @voidgiraffe, 2026-10-04
Source: @1a2b3c0967, 2026-10-04
The x16 mod is not required: Finishtilt14 runs the recipe without it and it runs well in TP or PP. Source: @finishtilt14, 2026-10-13
Mountains got P2P working, tried TP, and went back to PP4 because prefill on very long contexts is better. Source: @mountains1605, 2026-10-15
TP2 x PP2 (community flavour by cibernox on four x16 cards behind one PCIe switch): decode slower than TP4, prefill above 5k tok/s, so it sits between PP4 and TP4. He suggests it for people stuck on x4 links; not measured on x4. Source: @cibernox, 2026-10-07

Ivan's measurements, same bench on all three (single stream code / structured / prose, aggregate at 8 users, 17k-token prefill):

Layout	1 user	8 users	Prefill 17k	TTFT on a 49k prompt	KV pool at 262K
TP4	252 / 235 / 139	623 / 617 / 421	1.7k	18 s	1.20M
TP2 x PP2	171 / 146 / 98	567 / 503 / 383	4.2k	11 s	1.14M
TP1 x PP4	113 / 121 / 71	499 / 515 / 376	6.8k	not measured	2.09M

TP2 x PP2 suits long prompts with short answers (coding agents); PP4 suits prefill-bound batch work. Raising MAX_BATCHED to 8192 gave nothing on TP4 or TP2 x PP2 and cost KV. Source: @darkmaniac7, 2026-10-28 (lab notes)

On eight cards as a single instance (Ivan, cap 250 W): TP8 gave 95 / 105 / 56 tok/s for one user and 440 aggregate at 8 users with 554 tok/s prefill, because the 8-way all-reduce over Gen2 plus the socket hop dominates. TP4 x PP2 gave 103 / 145 / 38 for one user, dropped to 38 tok/s per stream at 2 users and only paid off at 32 users (492 aggregate) and in prefill (1,856). Two TP4 instances, one per CPU socket, is the better use of eight cards. Source: @darkmaniac7, 2026-10-27 (lab notes)

Gen2 x8 without P2P (Ivan, in an ESXi VM)

Ivan ran release 1.8.0 exactly as published (canada-quant checkpoint, DFlash2) on four 8 GB cards inside an ESXi guest at Gen2 x8 with no P2P (host-staged collectives), 180 W, on the recipe's own MiaAI protocol. Tok/s structured / code / prose at 74 SMs, the published figures in brackets:

Layout	1 user	2 users	4 users	8 users	Cold 32k prefill
TP4	361 / 284 / 199 (483 / 442 / 232)	441 / 371 / 273 (607 / 460 / 315)	545 / 468 / 401 (718 / 619 / 501)	697 / 587 / 529 (944 / 835 / 667)	1,633-1,753 (3,161)
PP4	201 / 149 / 97 (232 / 210 / 100)	309 / 223 / 183 (344 / 273 / 196)	386 / 353 / 300 (409 / 383 / 314)	664 / 548 / 477 (664 / 609 / 469)	6,834-6,977 (7,419)

PP4 lands close to the published numbers, so it is not link-bound. TP4 sits at about 0.75x because its all-reduce goes through host memory over Gen2 x8. Draft acceptance matched the published regime (95-99% structured, 74-89% code, 41-48% prose), so the drafter is not the gap. The 74-SM unlock (70 to 74) was decode-neutral within noise and worth 2-5% on PP4 prefill; clear the torch.compile and Triton caches when the SM count changes. Single-user prose varies about 10% boot to boot at temperature 0, other cells about 3%. TP4 settings there: VLLM_HOST_ALLREDUCE=1, VLLM_HOST_ALLREDUCE_MAX_SIZE=524288 (32 MiB was 9% slower on prefill), custom all-reduce off. A clock lock (nvidia-smi -lgc 1410,1410) did nothing: the cards already run 1,410 MHz under load. Recommendation: PP4 for long prompts, RAG, agents and many streams; TP4 for one interactive user with short prompts. Source: @darkmaniac7, 2026-10-19 (lab notes)

Drafter settings

On TP4 (Ivan's VM runs), DFlash2 with the release's adaptive table (7 / 5 / 3 drafts by load) is the best single setting for every workload and user count: it is worth 3.6x / 2.8x / 2.0x over no drafter for one user (structured / code / prose) and still 1.25-1.5x at eight users. An adaptive [7,7,5] table helps agent traffic (+14% structured, +7% code at 4 users) and costs prose. Without a drafter TP4 decodes 101 tok/s (9.9 ms per step), PP4 48 tok/s. Deeper drafts do not pay on these cards: each extra draft token costs about 1.1 ms of verify per step (21.2 / 23.4 / 25.6 ms at depth 7 / 8 / 10), and depth 8 trips an assert in the fork's fused sm_80 KDA decode kernel without a patch. Source: @darkmaniac7, 2026-10-20 (lab notes)

Planning a custom split

The model has 45 decoder layers (dense MLP at 0-2, MoE at 3-44), with 11 sparse MLA attention layers at 3, 7, 11 ... 43 and KDA linear attention elsewhere. VLLM_PP_LAYER_PARTITION needs one entry per stage summing to 45 (the MTP layer is not counted), and every pipeline stage must hold at least one MLA layer. Each rank plans its own KV pool and all ranks are cut to the smallest, so a split must balance memory as well as compute. Explicit PP and TP keys override LAYOUT. Source: @darkmaniac7, 2026-10-19 (lab notes)

Eight cards

Two instances of the recipe: about 1,600 tok/s aggregate on 8 cards (two PP4 groups, or two TP4 groups if x16 modded), release 1.6, power limit 180 to 200 W. Source: @impactic, 2026-10-20
Miséricorde expects 2,000+ if the cards are tuned and you can cool a 230 to 250 W power limit (not measured). Theory Source: @wakeupmbrc1, 2026-10-20
Con3po runs two GLM 5.3 Flash instances on 8 cards for an agent farm: 6 VMs and 8 sub-agents, about 12 s per action. Source: @con3po7016, 2026-10-16

KV cache and context

Vanilla TP4 startup line on cibernox's box: "GPU KV cache size: 1,204,813 tokens, Maximum concurrency for 262,144 tokens per request: 4.60x". He notes FP8 KV would roughly double it and that vLLM shares the cache, so five parallel 250k sub-agents rarely collide. Source: @cibernox, 2026-10-16
Dropping --max-num-seqs from 8 to 3 grew 1a2b3c0967's pool from 1.9M to about 3.5M tokens. He credits the 16-bit KV cache for the model staying sharp at high context; Morrowmake says research shows little loss from KV quantization but people report otherwise. Source: @1a2b3c0967, 2026-10-04
Source: @voidgiraffe, 2026-10-04
Con3po runs 1.3M max context stable on 4 cards. Source: @con3po7016, 2026-10-08
Miséricorde: 4 cards serve 8 users with W4A16 plus 4.5M tokens of KV, for example 9 threads at 500k each. Contested cibernox gets 1.2M on the stock recipe; the 4.5M figure needs a smaller sequence count or FP8 KV. Source: @wakeupmbrc1, 2026-10-16
Source: @cibernox, 2026-10-16

Power

The published numbers are at 180 W per card on stock cards; tuned cards could add maybe 20%. Source: @wakeupmbrc1, 2026-10-20
peter643 sees about 104 W per card while generating under a 180 W cap; at idle he downclocks and relaxes the HBM refresh to about 25 W per card. Source: @peter643, 2026-10-20
PixelML's power curve for this recipe (TP4, HBM 1,728 MHz; structured tok/s one user / eight users / four-card GPU power / tokens per watt): 165 W 418.6 / 822.4 / 650 W / 1.27; 150 W 418.0 / 816.5 / 582 W / 1.40; 140 W 412.7 / 814.2 / 537 W / 1.52; 120 W 374.4 / 756.5 / 476 W / 1.59; 110 W 340.1 / 706.8 / 436 W / 1.62; 100 W 291.9 / 627.4 / 398 W. Below 130 W the core clock falls fast. A driver reload or guest reboot resets the cap to the VBIOS default of 250 W, so re-apply it. Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @c133, 2026-10-20

See Cooling and power and Tools for caps and clock tools.

Known issues

Release 1.7 had a TP4 lost-context / endless-repetition bug when a speculative step ended exactly on a 1,152-token block boundary. Fixed in 1.7.2, along with GPU memory growth over hours and garbage output after a non-local cached prefix. Two-card owners hit by the repetition moved to the ExLlamaV3 builds; the fix only concerned 4 cards. Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06
Source: @mountains1605, 2026-10-14
GLM-5.3 always reasons. Set max_tokens for reasoning plus answer, or content comes back empty with finish_reason: "length". enable_thinking: false is ignored; use reasoning_effort (low, high, max).
From Z.ai's model notes: reasoning_effort on GLM-5.3-Flash accepts low, high and max and defaults to max when omitted or set to anything else; the chat template's clear_thinking defaults to false, so pass true for chat (toggling it mid-conversation costs the prompt cache). Source: repo github.com/zai-org/GLM-5 by zai-org on GitHub, posted by @cmdrwaka324, 2026-10-08
PP4 on real x4 links is not measured by the author (Mountains' numbers above are the field data).
The default DFlash2 drafter is CC BY-NC-ND 4.0, so the published configuration is non-commercial.
Updating is forward-only; older images may be retired.

Source: repo github.com/Morrowmake/glm53-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @ob88594, 2026-10-06

Contested Custom all-reduce on Gen2. The recipe turns vLLM's custom all-reduce on after the P2P check. On Ivan's box it first hung all workers at init for 20+ minutes; the cause was stale build locks in the compile cache. Once it started, on GLM TP4 it cost about 40% decode for 13% more prefill (205 vs 252 tok/s single, 380 vs 623 aggregate at 8), so his production runs NCCL only (VLLM_CUSTOM_ALLREDUCE_PCIE_P2P=0). On the Qwen3.8 Flash Next TP2 recipe the same switch helped instead (see Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2)). A/B it on your box. Source: @darkmaniac7, 2026-10-27 (lab notes)
Contested P2P on TP4. Morrowmake measured P2P helping on his EPYC host (about +11%) and his release numbers are with P2P verified. PixelML measured it hurting on their PLX + Broadwell host (-20 to -26% for one user, -38 to -48% at eight users, decode step 19.4 to 26.1 ms) and runs TP4 with P2P off (398.4 / 812.0 tok/s). ElectroHobo found the opposite on the same kind of host for MiMo V2.6 Flash, where P2P off is about 40% slower. Source: repo github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-18
Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @c133, 2026-10-20
canada-quant's GLM-5.3-Flash-DFlash2-G drafter does not load on this engine (indexer.k_cache page size error); NVFP4 GLM checkpoints do not run on sm_80 (SM12x FlashInfer only). Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @c133, 2026-10-20
MAX_SEQS=16 on TP4 crashed vLLM's CUDA-graph profiling ("operation not permitted"); keep 8 per instance. Source: @darkmaniac7, 2026-10-27 (lab notes)
MTP is broken on PP4 in this fork: acceptance 0.27 / 0.11 / 0.08 and 53 / 40 / 38 tok/s, worse than no drafter (48), because the MTP head gets the wrong inputs on the pipeline layout; the recipe lists MTP as unsupported there. PP2 with SPEC_MODE none or mtp failed to come up. The no-drafter mode fails the recipe's boot check (missing spec-decode counters): run it with BOOT_CHECK=0. Source: @darkmaniac7, 2026-10-19 (lab notes)
./start.sh stop needs the recipe's lifecycle lock and exits 1 without stopping anything if another command holds it. When passing a GPU subset to Docker, keep the inner double quotes (GPUS='"device=0,1,2,3"'), because Docker parses it as CSV. Do not benchmark while a CPU-heavy job runs on the host: layouts benched beside one read 180 / 126 / 94 instead of 247 / 233 / 138. Source: @darkmaniac7, 2026-10-28 (lab notes)
Running the recipe's checkpoint on the lazymio/vllm-backport engine instead does not work: the loader fails on the first dense layer (KeyError on layers.0.mlp.gate_up_proj.weight). Source: @darkmaniac7, 2026-10-28 (lab notes)

With other GPUs in the same job

Ivan runs the same engine across CMP and RTX PRO 6000 Blackwell cards in one vLLM job (sm_80 + sm_120):

Two CMPs as pipeline stage 0 (12 of 45 layers) plus two Blackwells as stage 1 (33 layers), PP2 x TP2: 349 / 261 / 180 tok/s for one user (structured / code / prose), 895 / 703 / 611 at eight users, the best eight-user aggregate he measured; 32k cold prefill in 5.95 s; a 996,784-token prompt prefilled in 206 s; KV pool 1.9M tokens at 262k (2.19M at 1M context in production). Two CMPs stay idle, because two cards cannot hold the 178 GB model.
Four CMPs plus two Blackwells: PP3 x TP2 gave 553 / 484 / 425 tok/s at eight users (code / JSON / prose) and a 7.1M-token KV pool; PP6 x TP1 prefilled 32k in 2.92 s (11,174 tok/s). Single-user decode was lower than a Blackwell-only stack.
It needs three small Python patches to the fork (device capability per rank, the MLA backend on sm_120, the host-staged all-reduce only on sm_80 TP groups) and launch settings: kda_prefill_backend: triton, CUTE_DSL_ARCH=sm_120a, FlashInfer autotune off, Blackwells as the last stage, every stage holding at least one MLA layer.
Raising HBM to 1,728 MHz on the two CMPs gave only +1.6-1.9% single stream in this hybrid, because the Blackwell stage dominates the step.

Source: @darkmaniac7, 2026-10-19 (lab notes)

Variants and related

Three cards: GLM 5.3 Flash on 3x 8 GB.
Two cards: GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 TP2), GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split), GLM 5.3 Flash on 2x 8 GB (Project Maya).
Engine notes: Engines. Quant choices: Quantization on the 170HX. All numbers: Benchmarks.
Abliterated: Ivan re-quantized dealignai's uncensored GLM-5.3-Flash FP8 checkpoint to the same W4A16 format for this engine (see Quantization on the 170HX#Heretic and abliterated variants); with the base-trained DFlash2 drafter it runs at 88-97% of the base's speed (one user 349 / 254 / 181 against 361 / 284 / 199). Source: @darkmaniac7, 2026-10-19 (lab notes)
Refusal steering: the Weightless GLP-44 vector (msuiche/weightless) is ported to the ExLlamaV3 two-card build, not to this recipe.

Sources

Repo: Morrowmake/glm53-flash-cmp170hx-recipe (MIT), last commit 2026-10-26. Posted on Discord by OB88594, Mountains and jjb. Source: @ob88594, 2026-10-06
Model: zai-org/GLM-5.3-Flash (MIT); vendor notes in zai-org/GLM-5.
Ivan's lab notes, 2026-10-27 and 2026-10-28.