DeepSeek V4.1 Flash on 6x 8 GB (TP6)

DeepSeek-V4.1-Flash-1017
Reported 6 x 8 GB
Quant	Official checkpoint: dense MXFP8 through an SM80 Marlin W8A16 path, routed experts MXFP4 through W4A16; KV FP8
Engine	Custom SM80 vLLM (patched from public commit 79a7108d9) reports 0.0.0; patches/public-base-to-runtime.patch (109 files)
Topology	TP6 (virtual-head padding), PP1
Context	1,048,576
Decode	78.55 tok/s
VRAM per card	64 GB, --gpu-memory-utilization 0.95
Flags	`--tensor-parallel-size 6 --kv-cache-dtype fp8 --block-size 128 --max-num-seqs 16 --max-num-batched-tokens 2048 --engram-config storage disk --speculative-config dspark 5; --hf-overrides virtual heads 96 / o_groups 12`
Tested	@wgkbfcp on 2026-10-19
Source	link
Confirmations	None yet.

A deployment record for DeepSeek-V4.1-Flash-1017 on six 64 GB cards at tensor-parallel 6, with the DSpark drafter at K=5, including a full snapshot of the author's patched SM80 vLLM source. It is an evidence package rather than an installer: the serving stack starts through a private local console that is not published, and the host is Windows with WSL2. The README is in Chinese. Infobox decode is end-to-end tok/s on code including reasoning tokens. Source: repo github.com/wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe by wgkbfcp on GitHub, posted by @dale063629, 2026-10-19
Source: @dale063629, 2026-10-19

Why TP6 is unusual

Contested vLLM requires the TP size to divide the model's attention head count; DeepSeek V4.1 has 16 heads per rank, so only TP 1, 2, 4 and 8 work, and six cards normally mean TP2 x PP3. Miséricorde: you cannot run TP6, and six cards may be worse than four. This recipe gets TP6 anyway by padding the real 64 heads / 8 groups to 96 / 12 (whole groups duplicated up to WO_A, the extra WO_B columns zeroed). The same override on unmodified vLLM does not work. Source: @greencandle9753, 2026-10-21
Source: @wakeupmbrc1, 2026-10-17
Source: repo github.com/wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe by wgkbfcp on GitHub, posted by @dale063629, 2026-10-19

Prerequisites

Six CMP 170HX at 64 GB, all at PCIe Gen2 x16 on the author's host (Driver unlock, 8 GB card).
Author's host: Threadripper PRO 3995WX, about 512 GiB RAM; Windows with WSL2 Ubuntu (256 GB WSL memory, swap 0), weights on an ext4 VHDX mounted read-only into WSL.
The official checkpoint deepseek-ai/DeepSeek-V4.1-Flash-1017 at 1bae0e10 (48 shards, 510,296,708,312 bytes). The Engram disk table (203 GB) is warmed into the Linux page cache.
build-essential: Engram's C helper is compiled at runtime.

Source: repo github.com/wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe by wgkbfcp on GitHub, posted by @dale063629, 2026-10-19

Steps

Not reproducible as published: the launch goes through a private console. The serve arguments it records:

vllm serve /model --tensor-parallel-size 6 --pipeline-parallel-size 1 --data-parallel-size 1 \
  --kv-cache-dtype fp8 --block-size 128 --max-model-len 1048576 \
  --max-num-batched-tokens 2048 --max-num-seqs 16 --gpu-memory-utilization 0.95 \
  --engram-config '{"storage":"disk","disk_threads":32}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":5}' \
  --enable-prefix-caching --enable-auto-tool-choice --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41 \
  --limit-mm-per-prompt '{"image":1048576}' --no-enable-flashinfer-autotune --mm-encoder-tp-mode data \
  --hf-overrides '{"num_attention_heads":96,"o_groups":12,"virtual_heads_from":{"num_attention_heads":64,"o_groups":8}}'

Environment: VLLM_PP_LAYER_PARTITION=40, DSV41_SM80_INDEXER_PREFILL_SEQ_CAP=1, DSV41_SM80_PREFILL_CHUNK_SIZE=2 (only 2 or 4 accepted; 2 only for this exact TP6 profile), DSV4_LOGITS_ROW_CHUNK=64, VLLM_DSV41_SPREAD_DECODE=2, VLLM_USE_BREAKABLE_CUDAGRAPH=1, VLLM_WSL2_ENABLE_PIN_MEMORY=1, VLLM_WORKER_MULTIPROC_METHOD=spawn, VLLM_ENGINE_READY_TIMEOUT_S=7200, PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False, HF_HUB_OFFLINE=1. CUDA graphs FULL_AND_PIECEWISE (largest capture seen: 192 tokens). Software: torch 2.13.0, triton 3.7.1, flashinfer-python 0.6.18, tilelang 0.1.12, transformers 5.17.0, Python 3.12.3, CUDA 13.0.2, TORCH_CUDA_ARCH_LIST=8.0. build/Dockerfile.source is a candidate rebuild that has never been run. Source: repo github.com/wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe by wgkbfcp on GitHub, posted by @dale063629, 2026-10-19

Measured numbers

2026-10-19, single request, Engram warm, prefix cache not reset, reasoning_effort=max, temperature 1: K=5 end to end including reasoning tokens 78.55 tok/s on code and 76.56 tok/s on writing (per request 61.26 to 84.76). Final-answer phase on code estimated at 188.25 tok/s (183.97 to 193.19) from stream timestamps over short windows. Not cold-start, not multi-user, not full 1M-context numbers.

K=5 against K=3, 24 samples: K3 gave +1.64% token rate but +7.69% completion time, below the 5% switch threshold, so K=5 stays. Draft acceptance K5 28.32% code / 27.34% writing against K3 41.42% / 43.36%. Code 6/6 passed (68 unit tests) in both arms; writing 4/6 (K5) against 5/6 (K3), non-blinded review. Source: repo github.com/wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe by wgkbfcp on GitHub, posted by @dale063629, 2026-10-19

The scheduler patch's notes record a measurement on 8 cards at PP8: a burst admitted in one step ran 274 tok/s aggregate (34 per stream) against 531 and 66 with requests spread over the pipeline ring (VLLM_DSV41_SPREAD_DECODE).

Other six-card results

TP2 x PP3 with the Deepinfer-SM80 patches: see DeepSeek V4.1 Flash on 6x 8 GB (Deepinfer-SM80 TP2 x PP3).
Con3po: 157 tok/s prose at concurrency 1 on six cards (quant not stated). Source: @con3po7016, 2026-10-21
DeeJay: about 67 tok/s decode on six cards, PP6, x4 links, native quant, still tuning. Source: @deejay9983, 2026-10-22
Reported OB88594: "6x single streams" run at about 5,000 tok/s prefill and 80 to 100 tok/s decode on DeepSeek V4.1 Flash 1017, and cloud services give 2 to 3x that. It is not clear whether he means six cards or six streams; engine, quant and layout were not posted. Summary of his posts; see Discord. Source: @ob88594, 2026-10-24
Source: @ob88594, 2026-10-24
Source: @ob88594, 2026-10-24
intervitens: a sixth card buys native-precision V4.1 Flash, with TP2 x PP3. Source: @intervitens, 2026-10-22
roboalchemist is packing an FP4-weight mix of 1017 sized for six cards, to be published on his Hugging Face page. Source: @roboalchemist., 2026-10-17

Known issues

Launch depends on a private console and host-specific guards (at least 400 GiB free commit before load, 99% residency per Engram table). Windows + WSL2 specific. No license for the recipe files (the vendored vLLM keeps Apache-2.0).
Engram warmed into the page cache is reclaimable, not locked; swap-out degrades speed.

Source: repo github.com/wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe by wgkbfcp on GitHub, posted by @dale063629, 2026-10-19

Sources

Repo: wgkbfcp/deepseek-v4.1-flash-1017-cmp170hx-recipe (translated from Chinese), last commit 2026-10-19. Posted on Discord by 大乐.
Related: Engines, Recipes, Benchmarks.