Running DeepSeek V4.1 Flash locally: hardware and measured speeds
More actions
Short answer. DeepSeek-V4.1-Flash-1017 runs on four 8 GB cards unlocked to 64 GB with a hybrid checkpoint (routed experts at about 3 bits in EXL3, everything else in DeepSeek's own FP8). The three recipes measured at 180 W per card report 258 to 297 tok/s for one user on code, 544 to 795 tok/s across eight users and 2,416 to 3,776 tok/s cold prefill, with a 262,144-token context. The official checkpoint, with no requantization, needs five or more cards: six and eight-card recipes serve it with a 1,048,576-token context at 78 to 161 tok/s for one user and up to 6,066 tok/s prefill. Every row below links to the recipe page with the exact commands.
Speeds are from each recipe's own benchmark, usually one user on code at temperature 0 with the DSpark drafter on. Read Known problems before choosing a four-card recipe: one of them miscopies names and paths at long context unless a fix is applied.
Hardware it needs
- Cards. Four unlocked cards (256 GB) hold the hybrid checkpoint. The official checkpoint (about 475 GiB, including the Engram lookup tables in its last two shards) runs with every expert in VRAM from five cards up. Cards must show 64 GiB each: see Driver unlock.
- Links. The four-card recipes use tensor parallel (TP4), which needs PCIe Gen2 x16 on every card (PCIe x16 capacitor mod). The GNmac recipe has a PP4 layout for narrower links, and the eight-card PP8 recipe runs on mixed link widths and treats P2P as optional.
- Host RAM and disk. The two Engram tables are about 189 GiB. Morrowmake's recipe keeps them in host RAM for its published numbers: about 240 GiB of RAM and about 680 GiB of disk. The GNmac recipe streams them from NVMe: 64 GB or more of host RAM and about 400 GiB of fast NVMe.
- Power. Most four-card numbers are at 180 W per card.
Source: repo github.com/Morrowmake/dsv41-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @lia_oc, 2026-10-24
Source: repo github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx by GNmac-lab on GitHub, posted by @greencandle9753, 2026-10-21
Measured configurations
One row per recipe page. "One user" is the recipe's headline single-stream decode figure; "Many users" is the total across concurrent requests; the prompt length behind a prefill figure is in brackets where the page gives it. Power caps, link widths and benchmark protocols differ between rows, so compare rows only after reading the recipe page. The status badge is the one each recipe page carries: Proven means a second person reproduced it.
| Cards | Recipe | Weights | Layout | One user (tok/s) | Many users (tok/s) | Prefill (tok/s) | Context | Power cap | Status | Measured by |
|---|---|---|---|---|---|---|---|---|---|---|
| 4 | DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake) | hybrid: EXL3 2.9 bpw routed experts, official FP8/BF16 for the rest | TP4 on x16 links, P2P optional; DSpark drafter | 297 | 795 at 8 users | 3,776 (2,048 tokens) | 262,144 | 180 W | Reported | voidgiraffe |
| 4 | DeepSeek V4.1 Flash on 4x 8 GB (GNmac) | hybrid: EXL3 routed experts (mostly 3 bpw), official FP8 for the rest | TP4 on x16 links with P2P (PP4 for narrow links); DSpark drafter | 258 | 544 at 8 users | 2,416 | 262,144 (up to 1,048,576) | 180 W | Proven | greencandle9753 |
| 4 | DeepSeek V4.1 Flash on 4x 8 GB (GNmac with copy fix and faster prefill) | as the GNmac recipe | TP4 on x16 links with P2P; DSpark drafter | 283 | 559 at 8 users | 2,732 (24k, 34k and 38k tokens, cold, median of 9) | 262,144 | 180 W | Reported | darkmaniac7 |
| 4 | DeepSeek V4.1 Flash on 4x 8 GB (CurtisAccelerate TC-MoE) | hybrid: EXL3 2.9 bpw routed experts, official FP8/FP4 for the rest | TP4 on x16 links, P2P off; DSpark drafter | 144.2 | 271.6 at 8 users | 1,523 (2,048 tokens) | 262,144 | 140 W | Reported | CurtisAccelerate |
| 4 | DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build) | hybrid: all expert layers at 3 bits, official FP8 dense tensors | TP4 on x16 links with P2P; no drafter | 105 | 482.4 at 8 users | 2,514 (24k to 38k tokens) | 262,144 | 200 W | Reported | alanoo |
| 4 | DeepSeek V4.1 Flash heretic on 4x 8 GB | the Morrowmake hybrid with the heretic patch applied | TP4; DSpark drafter | 158 | 323 at 8 users | 2,453 (17k tokens) | 262,144 | Reported | darkmaniac7 | |
| 2 | DeepSeek V4.1 Flash on 2x 8 GB (Project Maya) | 2-bit GGUF, experts tiered across VRAM, host RAM and NVMe | engine-managed tiering | 47 | 1,190 (4K tokens) | Reported | seriousrising | |||
| 5 | DeepSeek V4.1 Flash on 5x 8 GB (PP5 full residency) | official checkpoint (510 GB) | PP5 on x16 links; no drafter | not posted (30 to 40 in the repo's docs, 70+ in the Discord post) | 4,600 (60K tokens) | 1,048,576 | Reported | 344303947 | ||
| 6 | DeepSeek V4.1 Flash on 6x 8 GB (Deepinfer-SM80 TP2 x PP3) | official checkpoint | TP2 x PP3; DSpark drafter | 160.8 | 4,037 (16K tokens) | 1,048,576 | Reported | HalfVulpes | ||
| 6 | DeepSeek V4.1 Flash on 6x 8 GB (TP6) | official checkpoint | TP6 on x16 links; DSpark drafter | 78.55 (end to end, reasoning included) | 1,048,576 | Reported | wgkbfcp | |||
| 8 | DeepSeek V4.1 Flash on 8x 8 GB (Zanooda PP8) | official checkpoint | PP8, P2P optional; DSpark drafter | 117 | 560 at 8 users | 6,066 (104,881 tokens) | 1,048,576 | Reported | Zanooda | |
| 8 | DeepSeek V4.1 Flash on 8x 8 GB (Deepinfer-SM80 TP4 x PP2) | official checkpoint | TP4 x PP2; DSpark drafter | 153.35 | 620.22 at 12 users | 3,463 | 1,048,576 | Reported | HalfVulpes | |
| 4 plus one RTX PRO 6000 | DeepSeek V4.1 Flash on 4x 8 GB plus RTX PRO 6000 (ExLlamaV3 TP5) | EXL3 3.5 bpw | TP5 through host memory; MTP draft on the RTX PRO 6000 | 99.9 | 2,116 (26,296 tokens) | 1,048,576 | 150 W (CMPs) | Reported | anoane |
Sources for the rows (each recipe page carries its full credits):
Source: repo github.com/Morrowmake/dsv41-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @lia_oc, 2026-10-24
Source: repo github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx by GNmac-lab on GitHub, posted by @greencandle9753, 2026-10-21
Source: @darkmaniac7, 2026-11-09 (lab notes)
Source: repo github.com/CurtisAccelerate/DeepSeekv41-Flash-1017-EXL3-CMP170HX by CurtisAccelerate on GitHub, posted by @anon33anon, 2026-10-20
Source: @alanoo, 2026-10-27
Source: @darkmaniac7, 2026-10-29 (lab notes)
Source: @pcsv., 2026-10-25
Source: repo github.com/344303947/dsv41-flash-pp5-170hx by 344303947 on GitHub, posted by @dkpoulsen, 2026-09-22
Source: repo github.com/HalfVulpes/Deepinfer-SM80 by HalfVulpes on GitHub, posted by @reala7096, 2026-10-20
Source: repo github.com/wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe by wgkbfcp on GitHub, posted by @dale063629, 2026-10-19
Source: repo github.com/Zanooda/deepseek-v4.1-flash-cmp170hx by Zanooda on GitHub, posted by @gnasherrr, 2026-10-17
Source: repo github.com/anoane/09-deepseek-v41-flash-1017-exl3-tp5-1m by anoane on GitHub, posted by @anon33anon, 2026-10-20
Choosing between the four-card recipes
- Morrowmake has the highest published figures (up to 297 tok/s for one user on code, 795 at eight users, 3,776 tok/s prefill on a 2,048-token prompt) and its release 1.0.0 passed the lab's long-context copy test (0 of 29 paths wrong). It wants about 240 GiB of host RAM for those numbers. Source: repo github.com/Morrowmake/dsv41-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @lia_oc, 2026-10-24
Source: @darkmaniac7, 2026-10-29 (lab notes) - GNmac needs far less host RAM because it streams the Engram tables from NVMe, and a second person reproduced it. Release 1.2.0 as published miscopies names and paths at long context (see below). Source: repo github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx by GNmac-lab on GitHub, posted by @greencandle9753, 2026-10-21
- GNmac with the copy fix and faster prefill is release 1.2.0 plus the lab's fix and two prefill changes. Measured against the plain 1.2.0 image on the same four cards in the same session: cold prefill 2,732 against 2,258 tok/s (21% faster), 0 of 32 paths wrong on the copy test, decode unchanged. Source: @darkmaniac7, 2026-11-09 (lab notes)
- alanoo's fidelity build gives up single-user speed (105 tok/s on code, no drafter) for 3-bit experts on every layer and a 2.7M-token KV pool; eight-user throughput stays around 450 to 550 tok/s. Source: @alanoo, 2026-10-27
- Anemone is not a vLLM build: anon33anon's own C/CUDA engine for this model, released 2026-11-06 as anoane/anemone (v0.0.2-alpha, experimental), with no recipe page yet. Its README gives the author's own figures for four cards in TP4 with 2.9 bpw EXL3 experts: 337.2 tok/s structured decode for one user, 920.8 at eight users and 3,360 tok/s strict cold prefill, and it starts serving in about 30 s. Runtime Engram alone needs about 188.8 GiB of host RAM. See Engines#ExLlamaV3, TabbyAPI and EXL3. Source: @anon33anon, 2026-11-06
Source: repo github.com/anoane/anemone by anoane on GitHub
More than four cards
- Five cards hold the official checkpoint with every expert in VRAM at a 1,048,576-token context (PP5). Decode is not settled: the repository's docs give about 30 to 40 tok/s and the Discord post 70+. Source: repo github.com/344303947/dsv41-flash-pp5-170hx by 344303947 on GitHub, posted by @dkpoulsen, 2026-09-22
- Six cards at TP2 x PP3 measure 160.8 tok/s for one stream on code and 4,037 tok/s prefill at 16K. Eight cards at TP4 x PP2 measure 153.35 tok/s and hold 13.3M tokens of KV. Source: repo github.com/HalfVulpes/Deepinfer-SM80 by HalfVulpes on GitHub, posted by @reala7096, 2026-10-20
- Eight cards at PP8 have the best prefill of the DeepSeek recipes (6,066 tok/s on a 104,881-token prompt), 117 tok/s for one stream and a 6.2M-token KV pool, with no conversion of the official checkpoint. Source: repo github.com/Zanooda/deepseek-v4.1-flash-cmp170hx by Zanooda on GitHub, posted by @gnasherrr, 2026-10-17
- The four-card hybrid in one tensor-parallel instance on eight cards is slower than on four (measured 2026-11-09 on the GNmac build with the copy fix): 2,430 tok/s cold prefill against 2,755 the same night, one-user decode 17 to 20% lower, eight users about equal. Two four-card instances beat one eight-card instance. See DeepSeek V4.1 Flash on 4x 8 GB (GNmac with copy fix and faster prefill)#Limits. Source: @darkmaniac7, 2026-11-09 (lab notes)
Two cards
Most of the model does not fit on two cards. Project Maya keeps the hot experts and the KV cache in VRAM and moves the rest between host RAM and NVMe: the best documented two-card run is about 47 tok/s at temperature 1 with a 2-bit GGUF. Source: @pcsv., 2026-10-25
Known problems
- Long-context copy errors on the GNmac recipe as published. On the lab's test (75k to 92k-token turns that must copy image paths and names exactly), release 1.2.0 got 85% of named paths wrong. The cause is in that build's sparse-attention patch, not the quant: Morrowmake's recipe with the same expert files got 0 of 29 wrong. The fix is offered to GNmac-lab's repository as pull request 2, and a build with it is on DeepSeek V4.1 Flash on 4x 8 GB (GNmac with copy fix and faster prefill). Reported Source: @darkmaniac7, 2026-10-29 (lab notes)
Source: @darkmaniac7, 2026-11-08 (lab notes) - Reasoning is on by default on Morrowmake's recipe and arrives in the reasoning field.
max_tokensmust cover reasoning plus answer, orcontentis empty withfinish_reason: "length". Source: repo github.com/Morrowmake/dsv41-flash-cmp170hx-recipe by Morrowmake on GitHub, posted by @lia_oc, 2026-10-24 - Narrow links. On the GNmac recipe, TP4 on links narrower than x16 is very slow; use PP4 there. A startup hang at 100% GPU and low power is an all-reduce waiting on broken P2P. Source: repo github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx by GNmac-lab on GitHub, posted by @greencandle9753, 2026-10-21
- Engram tables in "host" mode. Contested Whether the Engram tables can fail with no visible sign:
- alanoo's build: left in "host" mode the tables load nothing and fail silently, with 15 of 20 exact copies and dropped underscores in identifiers while speed looks normal. His fix is to read the tables from a tmpfs. Source: @alanoo, 2026-10-27
- The lab's copy test on the GNmac recipe did not reproduce it: 120 of 120 lines and 360 of 360 underscores correct. Source: @darkmaniac7, 2026-10-29 (lab notes)
- Power on eight cards. On the PP8 recipe one of the author's cards fell off the bus (Xid 79) three times under sustained load at the stock 250 W limit and was stable at 180 W. Source: repo github.com/Zanooda/deepseek-v4.1-flash-cmp170hx by Zanooda on GitHub, posted by @gnasherrr, 2026-10-17
- Tool-call markup, loops and crashes have their own page: DeepSeek V4 and V4.1: tool-call leaks, loops and crashes. Drafter settings: Speculative decoding on the 170HX.
More: Known bad combinations, Instability under load, vLLM on sm_80: known bugs and patches.
Common questions
How much VRAM does DeepSeek V4.1 Flash need?
How many CMP 170HX cards do I need for DeepSeek V4.1 Flash?
How fast is DeepSeek V4.1 Flash on four CMP 170HX cards?
How much host RAM does DeepSeek V4.1 Flash need?
Can DeepSeek V4.1 Flash run at 1M context on CMP 170HX cards?
Does DeepSeek V4.1 Flash need the x16 mod or P2P?
Why does my DeepSeek V4.1 Flash build get file paths and names wrong in long prompts?
Related
- Recipe pages for this model are in the table above; Recipe finder filters all recipes by cards, model and engine.
- The older checkpoint: DeepSeek V4 Flash 0731 on 4x 8 GB (allover326 PP4 DSpark) and DeepSeek V4 Flash 0731 on 3x 8 GB (PP3 DSpark).
- Other model pages: Running GLM 5.3 Flash locally: hardware and measured speeds, Running GLM 5.3 locally: hardware and measured speeds, Running Qwen3.8 Flash Next locally: hardware and measured speeds.
- Recipes, Benchmarks, Engines, Quantization on the 170HX, Speculative decoding on the 170HX, Parallelism over the PCIe link.