The full GLM-5.3 now has a published recipe that spans eight CMP 170HX and two RTX PRO 6000 Blackwell cards: 753B total parameters (744B without the MTP draft layer) with a 524,288-token context, per @darkmaniac7 (lab notes, November 5, 2026). Its status is Reported: one tester so far.
What the recipe is
- The base is cebeuq's glm53-cmp170hx-recipe (MIT), which serves the same model on eight unlocked 8 GB cards at TP2 x PP4: his converted checkpoint, vLLM patches, int8-tile KV cache, PCIe all-reduce and MTP with PipeSpec. The lab ran it on its own eight cards and entered it as Partly reproduced (November 4, 2026).
- The two-host layout adds the Blackwell pair as the first pipeline stage (30 of the 78 layers) and runs the eight CMPs as four TP2 pairs of 12 layers, over one direct 100 GbE link.
- The lab's additions are public: the repository darkmaniac7/glm53-cmp170hx-blackwell-recipe, an overlay tarball on the wiki, and three engine fixes offered to cebeuq's repo as pull requests.
Measured
- One stream at temperature 0: 95 / 104 / 66 tok/s on structured, code and prose answers.
- Eight streams: 240 / 244 / 194 tok/s aggregate.
- Cold prefill: 3,400 tok/s at 17k tokens and 4,100 at 64k.
- KV pool: 817,024 tokens. Quality: 90.2% top-1 agreement with a TP8 reference.
- Against the eight CMPs alone, the Blackwell stage adds about 10% single-stream speed, 43% more KV cache, about 50% faster prefill at 17k tokens and 13 to 28% more aggregate at eight streams.
Structured output
- With the grammar fix for draft tokens, json_schema, named tool choice and json_object requests return valid JSON under speculative decoding, with 0 HTTP 5xx at one and eight streams.
- A second grammar fix makes constrained replies 1.75 to 1.85 times faster per stream at eight concurrent streams (lab notes, November 5, 2026).
- json_object replies that came back empty: 35 of 2,400 before a fix for replies that ended inside the model's reasoning, 0 of 2,000 after it (lab notes, November 5, 2026).
The general procedure for one model across two hosts, including the link and RDMA setup, has its own page: Running one model across two hosts.
Sources
- GLM 5.3 on 8x 8 GB plus 2x RTX PRO 6000 (vLLM TP2 x PP5)
- GLM 5.3 on 8x 8 GB (cebeuq glm53-cmp170hx-recipe) and its Confirmation 1
- Running one model across two hosts (100 GbE, RDMA, vLLM and SGLang)
- vLLM on sm_80: known bugs and patches
- cebeuq/glm53-cmp170hx-recipe and darkmaniac7/glm53-cmp170hx-blackwell-recipe on GitHub
- Status taxonomy