DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build)

DeepSeek-V4.1-Flash-1017
Reported 4 x 8 GB
Quant	Hybrid: all 40 expert layers at 3 bits (LibertAI 2.9 bpw pack with layers 18-22 grafted from the 3.5 bpw pack), official FP8 dense tensors
Engine	GNmac-lab recipe image 1.2.0 (vLLM with sm_80 patches) plus three patches GNmac-lab at 2ad5741ab1; image ds41-cmp170hx:1.2.0-fidelity
Topology	TP4, P2P on
Context	262,144
Decode	105.0 tok/s
Prefill	2514 tok/s
VRAM per card	64 GiB (64,911 MiB usable), GPU_MEMORY_UTILIZATION 0.98
Power cap	200 W
Driver	NVIDIA open 610.43.02 from a cmpunlocker fork with a P2P patch
Unlock build	cmpunlocker fork with P2P (without Morrowmake's TRAP31 patch)
Flags	`LAYOUT=tp4, MAX_MODEL_LEN=262144, MAX_NUM_SEQS=8, GPU_MEMORY_UTILIZATION=0.98, P2P=1, SPEC_TOKENS=0 (DSpark off), ENGRAM_STORAGE=ssd from a tmpfs`
Tested	@alanoo on 2026-10-27
Source	link
Confirmations	None yet.

A fidelity-first configuration of DeepSeek V4.1 Flash on 4x 8 GB (GNmac), aimed at agentic and concurrent work. It grafts 3-bit experts from LibertAI's 3.5 bpw pack into layers 18-22 of the 2.9 bpw pack (so every expert layer is 3-bit), turns the DSpark drafter off to free KV room, loads the Engram tables from a tmpfs, and runs at 0.98 memory utilization for a 2.7M-token KV pool. Single-user decode is lower than the speed-tuned builds; 8-user throughput stays around 450 to 550 tok/s. The author calls it a first rough version. Infobox decode is one user on code without DSpark. Source: @alanoo, 2026-10-27
Source: repo github.com/alanoo81/deepseek-v41-flash-4x-cmp170hx by alanoo81 on GitHub, posted by @alanoo, 2026-10-27

Prerequisites

Everything DeepSeek V4.1 Flash on 4x 8 GB (GNmac) needs: four cards at 64 GiB and 74 SMs on PCIe Gen2 x16 with working P2P (the author's nvidia-smi topo -p2p reports OK and NCCL passes on all four).
About 428 GiB of disk for the pinned weights (against 671.5 GiB for the Morrowmake recipe).
About 190 GiB of tmpfs for the Engram tables plus engine memory (the author's container limit was 460 GiB; 240 GiB is an inferred floor, not measured). Author's host: ASRock Rack ROMED8-2T, EPYC 7F52, 512 GB DDR4-3200 in 8 channels, 170tune +200 MHz at 1,500 MHz, Docker inside LXC with GPUs passed by CDI.
bc; serve.sh refuses to start if the GPUs already hold more than 2,000 MiB.

Source: repo github.com/alanoo81/deepseek-v41-flash-4x-cmp170hx by alanoo81 on GitHub, posted by @alanoo, 2026-10-27

Steps

The repo holds only patches against the GNmac recipe, pinned download and graft tools, test scripts and raw data. In outline:

Clone GNmac-lab/deepseek-v41-flash-1017-cmp170hx at 2ad5741ab1 (release 1.2.0) and apply recipe/serve.sh.patch, recipe/start.sh.patch and tools/fetch_official_nonexpert.py.patch (CDI GPUs, optional drafter, Engram dir mount, graft-aware expert-file regex, pinned official revision).
Download with pinned revisions: recipe/tools/download-pinned.sh (the 2.9 bpw pack, shards 20 to 24 of the 3.5 bpw pack, the official non-expert tensors and Engram shards). All 55 LFS files are checked against Hugging Face SHA-256.
Build the grafted hybrid: recipe/tools/make_hybrid_graft.py (profile k3; it checks K in the headers).
Copy the Engram tables into a tmpfs before start: recipe/engram-ram.sh.
Start with recipe/env.example values: LAYOUT=tp4, MAX_MODEL_LEN=262144, MAX_NUM_SEQS=8, GPU_MEMORY_UTILIZATION=0.98, P2P=1, PORT=8011, IMAGE=ds41-cmp170hx:1.2.0-fidelity (17.8 GB, about 20 minutes to build, first start about 10 minutes).
Expect the log line Built engram token map (129280 -> 99092 ids) for layers (1, 14).

Pinned weights: LibertAIDAI 2.9 bpw at 0be63c33 (196.3 GiB), LibertAIDAI 3.5 bpw at 04a492fc shards 00020 to 00024 (about 24 GiB), official deepseek-ai/DeepSeek-V4.1-Flash-1017 at 1bae0e10 (non-expert tensors by HTTP range, about 17.5 GiB, and Engram shards 00047-00048, about 189 GiB), Curtis White's quantization_config.json at b8b9e99. Source: repo github.com/alanoo81/deepseek-v41-flash-4x-cmp170hx by alanoo81 on GitHub, posted by @alanoo, 2026-10-27

Measured numbers

At 200 W, MiaAI-Lab protocol (temperature 0, thinking off, 400-token cap), tok/s structured / code / prose:

	1 user	8 users (aggregate)	Cold prefill
DSpark off (default)	105.4 / 105.0 / 104.8	559.7 / 482.4 / 449.2	2,514 (median of 9 runs, 24k to 38k-token prompts)
DSpark 5	369.8 / 267.3 / 200.1	structured +52%, code -7%, prose -2.5%

DSpark acceptance: 99.4% structured, about 70% code, 43 to 53% prose.

KV pool at 262,144 context on the all-3-bit build: 1,817,233 tokens at 0.95, 2,417,392 at 0.97, 2,717,420 at 0.98 (10.37 full sequences). The original 2.9 pack gives 3,347,338 at 0.97. DSpark costs 2.78 GiB per card (pool falls to 1,113,781 at 0.97). MTP is not available for V4.1 in this vLLM, only DSpark. Source: repo github.com/alanoo81/deepseek-v41-flash-4x-cmp170hx by alanoo81 on GitHub, posted by @alanoo, 2026-10-27

Quality and fidelity

Graft effect by paired NLL on 64 frozen 1,024-token windows per set, bootstrap 95% CI: perplexity -1.80% overall [-2.00, -1.60], -4.45% on French [-4.97, -3.93] (64 of 64 windows better), -0.70% English, -0.18% code; it costs about 1.98 GiB per card.
Moving layers 0, 1, 3 and 6 to 4 bits gave +0.08% [-0.01, +0.18] for 1.58 GiB per card and was dropped. The full 3.5 bpw pack needs about 9.8 GiB more per card and does not fit. Miséricorde: no change from 4-bit layers is expected if the original MXFP4 parts are kept; those help decode speed more than quality. Source: @wakeupmbrc1, 2026-10-27
Copy test of 40 random identifiers per list: 300/300 exact over 100 lists x 3 passes (cold, prefix-cached, second turn). 5 tasks x 5 solo repeats byte-identical. Prompt logprobs bit-identical across restarts and between 0.97 and 0.98. Two waves of 8 concurrent unique 100k-token prompts completed 16/16.
A comparison against Morrowmake's recipe (from reading it, not running it) is in docs/comparison-morrowmake.md: much better perplexity in French, roughly equivalent in English. A 5-agent, 2-hour real-world run succeeded.

Source: repo github.com/alanoo81/deepseek-v41-flash-4x-cmp170hx by alanoo81 on GitHub, posted by @alanoo, 2026-10-27
Source: @alanoo, 2026-10-27

Known issues

Engram trap. Leaving ENGRAM_STORAGE empty ("host" mode) reserves 2 x 23.6 GiB pinned per rank but never fills it: the hybrid index lacks engram.embed.* and vLLM skips its missing-weights check when a quantization method is set. Symptoms: 15/20 exact copies, dropped underscores in identifiers, output changing between runs at temperature 0, while decode speed and the smoke test look normal. The log line "offloaded to pinned" means host mode. Fix: ENGRAM_STORAGE=ssd reading from a tmpfs prepared by recipe/engram-ram.sh.
Margin at 0.98: 63,580 MiB peak per card under the 8 x 100k stress (about 1.3 GiB left); prompt_logprobs on 1,024-token windows peaked at 64,676 MiB (0.23 GiB left), so use 0.97 or windows of 512 tokens or fewer for NLL work.
Engine numerics: GNmac's --kv-cache-dtype fp8_ds_mla writes the DeepSeek-V4 record (448 FP8 dims, one scale per 64) because V4.1's nvfp4_ds_mla format needs SM100; Morrowmake's BF16 KV reproduces the reference's FP8/FP4 rounding bit for bit.
The engine image contains AGPL-3.0-only vllm-exl3 code (section 13 applies if served over a network). Single machine, single set of measurements, at 200 W (not directly comparable with 180 W numbers). Test script messages are in French.