GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split)/Confirmation 1
From CMP Wiki
More actions
| Confirms | GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split) |
|---|---|
| Result | Reproduced |
| By | @nestaa51 |
| Date | 2026-10-09 |
| Cards | 8 GB |
| Numbers | 96.2 tok/s on easy text, 34.2 tok/s on hard text (decode, one request). Prefill 2 to 5% slower than his TP build. |
Run on two x16-modded 8 GB cards (Threadripper 3970X host) as the baseline for his tensor-parallel extension of this repo, which was 28% faster. Source: repo github.com/alexvnesta/glm53-flash-cmp170hx-tp by alexvnesta on GitHub, posted by @nestaa51, 2026-10-12