Qwen3.8 27B on 1x 8 GB (W4A16 vLLM)/Confirmation 2
From CMP Wiki
More actions
| Confirms | Qwen3.8 27B on 1x 8 GB (W4A16 vLLM) |
|---|---|
| Result | Reproduced |
| By | @kellertreppe123 |
| Date | 2026-10-05 |
| Cards | 8 GB |
| Numbers | About 170 tok/s decode at 262k context and 60 to 70 tok/s nearing long context, against the post's 160+ on short context and 70+ near the long end. |
He copied Miséricorde's r/LocalLLM recipe and ran it on one 64 GB card, mentioning 170tune alongside it. His own AutoRound quant of the huihui abliterated variant only reached 60 to 70 tok/s and drops somewhat in long context, which he attributes to his quant. Source: @kellertreppe123, 2026-10-05
Source: @kellertreppe123, 2026-10-05
Source: @kellertreppe123, 2026-10-05