DeepSeek V4 and V4.1: tool-call leaks, loops and crashes

DeepSeek V4 Flash (the 0731 checkpoint) and V4.1 Flash, served with the sm_80 vLLM forks on the CMP 170HX, share a family of agentic failures: tool calls that come back as raw DSML markup, reasoning text that leaks into the reply and is then sent back as conversation history, "Let me check..." loops that never make the call, and, deep into long contexts, wrong tokens and engine crashes. Most leaks are parser and chat-template gaps with published patches; the repetition loops past about 100k tokens are mitigated, not solved. Recipes: DeepSeek V4 Flash 0731 on 4x 8 GB (allover326 PP4 DSpark) for V4 and the V4.1 pages under Related. Engine-wide bugs are on vLLM on sm_80: known bugs and patches.

First checks

Reported Before debugging a leak or a loop:

Serve with --enable-auto-tool-choice --tool-call-parser deepseek_v4 --reasoning-parser deepseek_v4 (V4.1 builds use the deepseek_v41 parsers). Without the reasoning parser, thinking text arrives inside content with nothing marking it as reasoning. Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub
Source: repo github.com/Zanooda/deepseek-v4.1-flash-cmp170hx by Zanooda on GitHub
Have the agent client send chat_template_kwargs with "thinking": true and an explicit reasoning_effort on every request, not only the first; Schaka's repo recommends max for agentic sessions. Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub, posted by @schakaqt, 2026-09-02
Check which field your client reads: these builds return reasoning in reasoning, not reasoning_content (see Reasoning returned in the wrong field, below). Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub
Run a patched parser: Schaka's series (table at the end of this page) for V4 Flash 0731; for V4.1, the vLLM PR #52645 parser commits plus the empty think-tag fix. Source: @schakaqt, 2026-09-11
Source: @zanooda, 2026-09-11
For long conversations set DSV4_LOGITS_ROW_CHUNK=64 (see Crashes, below). Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub
After changing the server config, start a fresh agent session: the poster of one long-context loop report later blamed a session started before his configs were right. Source: @diabeetee, 2026-08-06

Tool calls and reasoning leaking into the reply

DSML tool-call markup returned as raw text (open question)

Josh reported DeepSeek output where the tool call came back as raw DSML markup (<｜DSML｜tool_calls> / <｜DSML｜invoke name="bash"> ...) instead of being parsed into a tool call, and asked what he needed to update (2026-08-29). Nobody answered in the thread. A tool-call parser or chat-template mismatch in the serving stack is the likely cause but unconfirmed; Schaka later cited open vLLM issues about DSML leaks (early-September loops, below). The next entry describes the parser gap that the published patches close.

Theory
Source: @_____josh_____, 2026-08-29

V4 Flash 0731: orphan invoke blocks and wrapper variants (Schaka patches 0009 to 0018)

Schaka's repo reports that at long context DeepSeek V4 sometimes omits the opening <｜DSML｜tool_calls> wrapper while still writing a well-formed <｜DSML｜invoke ...> block. The stock parser entered tool mode only on the wrapper, so the whole call leaked to the user as raw text (vLLM issue #48931). Patch 0009, a port of vLLM PR #49117, starts a tool call from a bare invoke marker. Patches 0010 to 0015 harden that path so prose that quotes the markup does not become a call. Patch 0016 covers invokes written before the thinking block closes: 0731 starts every turn in the reasoning state, so those leaked as raw DSML or were dropped. Patch 0018 accepts the <｜DSML｜_tool_calls> (leading underscore) wrapper the checkpoint occasionally writes. Per-patch details are in the table at the end of this page.

Reported 8 GB
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub, posted by @schakaqt, 2026-09-02

V4.1 Flash: reasoning and tool-call leaks fixed with PR 52645 commits and an empty think-tag fix

Schaka's main problem with DeepSeek V4.1 Flash (2026-09-11) was reasoning loops around tool calls and think tags that were sometimes not closed, or not opened at all. Reasoning text then landed in the main agentic context, was sent back, poisoned the context and wasted tokens. He had fixed this reliably for V4.0, but 4.1 behaves differently: it tried tool calls inside thinking blocks, got stuck and escaped the reasoning parser, and he broke his setup while trying to fix the thinking parser. Zanooda had fixed leaking tool calls, common with several parallel agents on V4 Flash, with commits 1 to 3 of vLLM PR #52645, reused as patches 0003 to 0006 in patches/v41 of his repo (the repo's patches/README.md lists v41/0003 to 0006 as PR commits 1 to 4). Schaka still saw escapes; the rest was empty thinking tags breaking vLLM parsing in combination with what the harness sends back, fixed by suppressing an early </think>. Result: no escapes in more than 2M output tokens on 6 cards, multi-session, and Zanooda adopted the </think> patch too.

Proven
Source: @schakaqt, 2026-09-11
Source: @schakaqt, 2026-09-11
Source: repo github.com/Zanooda/deepseek-v4.1-flash-cmp170hx by Zanooda on GitHub, posted by @zanooda, 2026-09-11

V4.1 Flash: later parser patches for misspelled wrappers, hybrid XML calls and headless calls

Zanooda's V4.1 patch series (last commit 2026-10-19) adds four more parser fixes, according to its patches/README.md:

v41/0012 accepts the near-miss wrapper spellings <｜DSML｜calls> and <｜DSML｜_calls> that the model writes near the context limit, so the misspelled closing tag no longer leaks into the reply.
v41/0014 heals hybrid XML/DSML tool calls. In long agentic sessions the model sometimes opens a call in a generic XML idiom (<invoke name="...">, or the tool name fused into <parameter name="TOOLNAME">) and returns to DSML by the close. Without the fix the whole call leaks, the harness echoes that turn back, and the model imitates its own broken format, so every later call in the conversation leaks too.
v41/0017 recovers headless calls where the model drops both the invoke opener and the tool name; the tool is inferred only when exactly one declared tool covers every parsed parameter, otherwise the block stays text.
v41/0020 syncs the V4.1 prompt encoder with the 1017 reference (partial port of vLLM PR #62196). The old rule dropped the reasoning before a tool call and emitted an empty think block after every tool result, the state in which V4.1 drifts into ad-hoc XML tool calls.

Reported
Source: repo github.com/Zanooda/deepseek-v4.1-flash-cmp170hx by Zanooda on GitHub

Reasoning returned in the wrong field

On these vLLM builds the reasoning text comes back in a response field called reasoning, not reasoning_content, the name DeepSeek's own API and most providers use. A client that reads only reasoning_content sees no thinking and cannot tell a thinking run from a non-thinking one. Schaka's patch 0023 also emits reasoning_content as a copy. The repo suspects, without a packet capture, that the mismatch is why opencode showed a whole think block as plain output and missed a tool call made inside it; once that text is folded into content and sent back, the chat template cannot drop it as stale thinking, and the damage can compound over a session. Separately, the think delimiters are special tokens that are stripped on decode, so without --reasoning-parser deepseek_v4 the reasoning lands inside content unmarked; the flag is inert when thinking is off.

Reported
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub, posted by @schakaqt, 2026-09-02

Thinking loops and repetition

V4 Flash: thinking loops and tool-call breakage from about 100k tokens (early September)

Schaka saw broken tool calls and repetition with DeepSeek V4 Flash from about 100k tokens on, especially with DSpark, on a stack based on the A100 fork of vLLM, with both OpenCode and DSH showing it, and pointed to many open vLLM issues about DSML leaks for DeepSeek (2026-09-01). The model repeatedly announced a tool call ("Let me check run progress.") without making it, or emitted corrupt tool tags. His workaround at the time detected the loop, cancelled the stream and spliced in a "continue" token; DSpark doubles decode locally but makes the think-looping worse. giaki3003 recommended provider pinning on OpenRouter, and Ryko said loops in big models usually come from too-low temperature (for example 0), a bad quant or a bad engine, and should be fixed rather than hidden. The hardware for these runs was not stated. On 2026-09-02 Schaka got rid of the thinking loops: the cause was mostly agents leaking thinking back into the context because vLLM does not expose it correctly. On 2026-09-03 he published his patches as Schaka/deepseek-v4-cmp170hx, said the base that robbbbbb7973 had forked from causes reasoning loops, and offered to merge the vision work.

Reported
Source: @schakaqt, 2026-09-01
Source: @schakaqt, 2026-09-01
Source: @giaki3003, 2026-09-01
Source: @vektor_prime, 2026-09-01
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub, posted by @schakaqt, 2026-09-02

V4 Flash 0731: "Let me..." loops above 120k to 140k context, fixed by capping the agent client at 115k

machinezero had run DeepSeek V4 Flash 0731 on 4 unlocked 8 GB cards for 4 days (2026-08-11) when he saw rambling "Let me...." output in the Pi Code harness after 3 to 5 hours; restarting the session fixed it without touching vLLM, and quark2world agreed it was a harness or software issue rather than the cards. Testing from 128k to 1M context, he found the model falls into these loops once context reaches 120k to 140k; compaction never fixed it, only a new session did. His fix: keep vLLM serving 1M context but set the agent client (Oh my Pi) to 115k with auto-compaction as early as 85%, which makes it usable in an endless session. He runs a fork of allover326/deepseek-v4-cmp170hx built with CUDA 13.3; DeeJay notes decode drops as context grows.

Reported 8 GB
Source: @machinezer0, 2026-08-11
Source: @quark2world, 2026-08-11
Source: @machinezer0, 2026-08-18
Source: @deejay9983, 2026-08-18

V4 Flash 0731: loops before tool calls at long context, a bug, a stale session or a model limit

Contested diabeetee ran the full-precision DeepSeek-V4-Flash-0731 safetensors on vLLM at 1M context, using DeeJay's latest commit and the pi harness (2026-08-06). About 100k tokens into a session the model looped repeatedly before making tool calls, while llama.cpp with the official GGUF had not hallucinated this badly. The views, in the order posted:

diabeetee later blamed a stale session started before his configs were right, since a fresh one ran to 300k with no repeats. Source: @diabeetee, 2026-08-06
giaki3003 suggested checking the vLLM tool-calling parser. Source: @giaki3003, 2026-08-06
DaLZy floated undetected HBM errors (no ECC) as an untested cause; see Memory errors and ECC. Source: @dalzyyy, 2026-08-06
DeeJay ran a 1M-context stress test with random prompts and tool calls and got 100% clean results. Source: @deejay9983, 2026-08-06
bayley_w sees it as a limitation of small sparse-attention models at long context ("this isn't a bug"; MiMo-2.5 did the same). Source: @bayley_w, 2026-08-06
Impactic suggested the chat templates in the model's encoding/ folder on Hugging Face. Source: @impactic, 2026-08-06

Repetition tripwires and in-turn recovery (Schaka patches 0020 to 0022)

Schaka's repo detects and recovers degenerate generation in the scheduler. Patch 0020 catches a "soup" of invented control tags (<system-reminder>, </assistant>, <tool_calls and similar, matching neither DSML nor a real protocol) and exact repeated lines, plus a fuzzy detector for varied wording ("Let me find/check/search..."). Patch 0021 splices a short nudge into the same turn and lets generation continue: after a plain abort, opencode's agent loop simply stops, because the abort looks like a normal end of turn, while typing "go" into the stuck conversation reliably unstuck the model. The retry budget is DSV4_NUDGE_MAX_RETRIES (default 2); DSV4_NUDGE=0 turns recovery off. In live tests past about 100k context the nudge fired, but the model kept mutating into new short repeats ("Let me check", "Check the run", "Check.") and used up the budget. Patch 0022 adds an opt-in DSV4_RECOVERY_MODE=reopen that splices a real turn boundary (end of the assistant turn, a user turn saying "go", a new assistant turn) instead of free text; it was untested when published. Setting top_p 0.95 server-side (the official 0731 agentic guidance) together with the reasoning parser did not fix the loops past 100k (tested 2026-09-01); the repo points to an unresolved llama.cpp report of the same shape (issue 26694: long agentic chats, local serving only, suspected loss of the turn boundary in chat-template or stop-token handling). Loops past about 100k context are mitigated, not solved.

Reported 8 GB
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub, posted by @schakaqt, 2026-09-02

Client side: send reasoning_effort on every request

In live testing on Schaka's served image, pinning chat_template_kwargs: {"reasoning_effort": "high"} on every request measurably reduced how often loops start, not just how fast the tripwires catch them; a session that never sends an explicit effort can drift over many turns into the self-reinforcing "Let me..." phrase lock. The repo cites wtdcode/vllm-backport with the same finding. A discussion on the model's Hugging Face page (DeepSeek-V4-Flash-0731 discussion 39) reports that low and high can render an empty thinking prefix on some vLLM versions; the empty think blocks pile up in the history and the thinking rate fell from 63% to 3% over about 100 turns. Switching to {"thinking": true, "reasoning_effort": "max"} resolved it for that reporter; the repo had not verified this on its own build. In opencode the setting goes under the model's options.chat_template_kwargs in opencode.json, not in a system prompt.

Reported
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub, posted by @schakaqt, 2026-09-02

Long-context garbage around 200k on V4.1 (and on GLM), no fix posted

Con3po sees DeepSeek V4.1 start flipping through pages and acting randomly during computer use around 200k context; /clear restores normal behavior. His KV cache is 8-bit (b8, 6 cards), with a light abliteration at 0.5. No layouts were given and there was no follow-up.

68x: this is usually a KV issue; he hit the same garbage output with EXL3 GLM when context got too long, and suggests compaction in the harness. Source: @68x0, 2026-10-26
segmond: that means inference has a bug; compaction is a band-aid, not a fix. Source: @segmond, 2026-10-26

Nobody in the guild has checked whether the 200k failures are the same defect as the pipeline-parallel hand-off bugs tracked as vLLM PR #60477 and issue #63170 (see vLLM on sm_80: known bugs and patches). Earlier long-context loop reports are the V4 Flash entries above; speculative-decoding speedups also collapse as context grows (Speculative decoding on the 170HX).

Reported
Source: @con3po7016, 2026-10-26
Source: @darkmaniac7, 2026-10-26 (lab notes)

Wrong tokens and copy errors

DSpark tool-call corruption from NaN logits rows (patches 0007 and 0008)

Schaka's patch 0007 adds a NaN to -inf guard before tl.argmax in the rejection sampler, a port of upstream vLLM #50183. Without it, an all-NaN target logits row makes the argmax return an index in the padded region, and an arbitrary token is committed as if verified. On sm_80 with an fp8 KV cache this shows up as DSML and tool-call corruption under DSpark greedy speculation; the investigation the repo links (allover326/deepseek-v4-cmp170hx issue 9, run on 2x 8-GPU A800 at TP4 with expert parallel) measured 35-65% corruption without the guard and 56 of 56 clean structured tool calls, streaming included, with it. Patch 0008 clamps three tile-local argmax sites to vocab_size - 1, because an out-of-vocab index otherwise reaches the V4 hash-MoE router and faults every rank. Zanooda's V4 series carries the same NaN guard as its patch 0008 and notes that the damage is invisible in prose while it shreds structured output such as tool calls and digit runs.

Reported
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub, posted by @schakaqt, 2026-09-02
Source: repo github.com/Zanooda/deepseek-v4.1-flash-cmp170hx by Zanooda on GitHub

GNmac TP4 port corrupts long-context verbatim copying (85% of paths wrong at 75k to 92k tokens)

In lab notes (2026-10-29), a roleplay-app workload on DeepSeek V4.1 Flash served by the GNmac TP4 port (DeepSeek V4.1 Flash on 4x 8 GB (GNmac)) mangled character names and image paths on real 75k to 92k-token turns: 85% of named paths wrong, against 0% on GLM-Flash. The lab reproduced it with a six-context byte-for-byte scorer and bisected one change at a time on stock dense weights:

GNmac production (2.9 bpw EXL3 experts, fp8_ds_mla KV, DSpark): 85% wrong.
Prompt cut to the last 5 turns: 33% (so the error is long-range).
BF16 KV cache: 100%. Engram synchronous path (async, overlap and prefetch off): 100%.
The same 1017 model at full precision through a hosted API: 0%.
The Morrowmake dsv41 recipe v1.0.0 with the same 2.9 bpw expert files: 0% (0 of 29, no name misses); see DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake).

Also ruled out: the heretic patch, the app's sampler, DSpark, the prompts and the detokenizer (logprobs show the model ranks the wrong token first; img after /w is not in its top 5). The defect is in GNmac's sm_80 attention or kernel stack, not the quant; a bigger quant or more cards will not fix it. A separate Engram copy test on GNmac did not reproduce the silent Engram failure in the next entry.

The same symptom class was reported earlier on another stack: Schaka's DeepSeek-V4.1 setup started mangling file paths into similar-looking wrong names on 2026-09-11, and bayley_w identified it as token corruption rather than hardware memory corruption. Which engine Schaka ran that day is not stated, so whether it was the same defect is unknown. bayley_w's bisect method for token corruption is on vLLM on sm_80: known bugs and patches.

Proven 8 GB
Source: @bayley_w, 2026-09-11
Source: @schakaqt, 2026-09-11
Source: @darkmaniac7, 2026-10-29 (lab notes)

Engram host mode on V4.1 Flash reportedly fails silently (copy test 15/20); not reproduced on GNmac

Contested 8 GB Whether Engram can fail without any visible sign:

alanoo's README (commit da967740, 2026-10-27; DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build)): the Engram host mode failed silently, and layers 1 and 14 most likely acted as identity functions. Symptoms: 15/20 exact copies, identifiers with dropped "_" separators, output changing between runs at temperature 0. "Decode speed is essentially unchanged without Engram", so speed will not reveal it. This matches wakeupmbrc1 (2026-10-17): dsv41 calls PLE in just two layers, layer 1 and layer 14. Test: about 40 random identifiers in about 20 lists, copied back. Settings to check: DSV41_ENGRAM_STORAGE=ssd on GNmac and an explicit --engram-config storage setting on Zanooda's build; Engram reads are latency bound ("nvme overhead will be like 0.2-0.3ms per round"). Source: repo github.com/alanoo81/deepseek-v41-flash-4x-cmp170hx by alanoo81 on GitHub, posted by @alanoo, 2026-10-27
Source: @wakeupmbrc1, 2026-10-17
The lab's copy test on GNmac (2026-10-29): 120/120 lines, 360/360 underscores, and a byte-identical repeat at temperature 0 with and without DSpark, so the failure did not reproduce there. Whether it is specific to the "host" storage mode or to alanoo's build is open. Source: @darkmaniac7, 2026-10-29 (lab notes)

Crashes and patches that fail to apply

Stock vLLM with V4 Flash: "hyperconnection.hpp:56 Unsupported architecture"

machinezero ran stock vllm serve with deepseek-ai/DeepSeek-V4-Flash (TP4, --enable-expert-parallel, max-model-len 393216, fp8 KV, block size 256, deepseek_v4 tokenizer and parsers, FLASHINFER attention, FULL_AND_PIECEWISE cudagraphs), which failed with Assertion error (csrc/apis/hyperconnection.hpp:56): Unsupported architecture (2026-08-09). robbbbbb7973 pointed him to allover326/deepseek-v4-cmp170hx (DeeJay's repo). machinezero got it running with TP after a lot of back and forth with Grok: spec decode mean acceptance length 3.68, accepted throughput 114.79 tok/s, per-position acceptance 0.809/0.667/0.527/0.396/0.277, average 53.5%. DeeJay: the repo works as published, so do not use Grok, and his fork is built for PP plus DSpark, not TP.

Reported
Source: @machinezer0, 2026-08-09
Source: @robbbbbb7973, 2026-08-09
Source: @deejay9983, 2026-08-09

deepseek-v4-cmp170hx patch 0006 fails to apply on newer vLLM

aaron found that patch 0006 of the deepseek-v4-cmp170hx repo (the context-ceiling fix in sparse_attn_indexer that enables 1M context) no longer applies: vLLM commit fded064 restructured the prefill path with query sharding and removed _prefill_topk_needs_torch_fallback() (2026-08-05). diabeetee hit 2 out of 5 hunks FAILED -- saving rejects to file vllm/model_executor/layers/sparse_attn_indexer.py.rej and worked around it by editing lines 508 to 518 of sparse_attn_indexer.py to skip ops.top_k_per_row_prefill when logits.shape[0] == 0, otherwise passing logits.shape[0] as the row count. Impactic's earlier attempt had also failed on patch 0006. DeeJay updated the repo's patches (0006 is the one that changed) on 2026-08-06 and asked people to open GitHub issues.

Reported
Source: @aaron0010, 2026-08-05
Source: @diabeetee, 2026-08-05
Source: @deejay9983, 2026-08-06
Source: @impactic, 2026-08-06

3-card V4: "RPC call to sample_tokens timed out" on the first token

On a 3x 170HX DeepSeek V4 Flash setup, aaron hit the EngineCore error TimeoutError: RPC call to sample_tokens timed out. while generating the first token (2026-08-09). He asked flipnburn for his final command to compare, and flipnburn answered with the Claude-written README of his working 3-card setup (deepseek-v4-flash-on-3x-cmp170hx.md). No root cause was posted. For comparison, the deepseek-v4-cmp170hx README states that 3 cards need VLLM_PP_LAYER_PARTITION=15,15,13, because the last pipeline rank also carries lm_head and the DSpark drafter; the thread does not say whether aaron's command set it. Working 3-card setups are on DeepSeek V4 Flash 0731 on 3x 8 GB (PP3 DSpark).

Reported
Source: @aaron0010, 2026-08-09
Source: @flipnburner, 2026-08-09
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub

A daily illegal memory access with DSv4, DSpark and pipeline parallel

machinezero's 170HX rig was stable for 4 weeks, then began crashing about once a day; the only recent changes were "the BAR1 commit" and switching the model from DSV4F to dsv4fv (2026-09-10). The vLLM log showed rank 3 hitting a real CUDA illegal memory access (cudaErrorIllegalAddress) during DSpark speculative verify plus the pipeline-parallel sampled-token broadcast, not an OOM (the KV use was tiny). His AI assistant's suspects: a DSpark draft token out of vocabulary used as a table index; NaN or garbage top-k indices in the DSv4 sparse prefill or decode; or the DeepGEMM, FlashMLA or sparse decode kernels under 1 to 7 token verify shapes, with the PP broadcast only reporting the fault. reala asked about a memory test; none had been run. Patches 0008 and 0019 of Schaka's series guard two of those paths (an out-of-vocab argmax, and the DSpark out-of-vocab draft sentinel); nobody reported trying them against this crash.

Reported
Source: @machinezer0, 2026-09-10
Source: @reala7096, 2026-09-10
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub

Accumulating conversations crash earlier than one-shot prefills: DSV4_LOGITS_ROW_CHUNK=64

The deepseek-v4-cmp170hx README (allover326's 4-card measurements, kept in Schaka's copy): with DSV4_LOGITS_ROW_CHUNK=128 a one-shot prefill reaches 1,047,736 tokens, but a real multi-turn chat on the same config dies at about 718k to 733k with a CUDA illegal memory access in _top_k_per_row_prefill_torch, always on the same PP rank (reproduced twice). At 64 a 405-turn conversation reached 1,002,852 tokens clean. The crash is specific to the prefix-cached path, which needle tests never exercise. Use 64 for conversational and agent workloads.

Reported 8 GB
Source: repo github.com/allover326/deepseek-v4-cmp170hx by allover326 on GitHub
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub

Grammar-constrained tool calls killing requests and workers under DSpark (patch 0019)

Schaka's patch 0019, squashed from five commits of ChinaBoy0618/deepseek-v4-cmp170hx, keeps grammar-constrained decoding failures under DSpark from killing the request or the worker: an FSM rejection abandons enforcement instead of terminating (the tool call can still be recovered from the unconstrained tail), speculative blocks are grammar-checked before commit rather than after, DSpark's out-of-vocab draft-buffer sentinel is truncated instead of crashing a worker, and a scheduler and worker truncation desync behind two production EngineDead incidents in that fork is fixed. Zanooda's V4.1 series ports upstream vLLM PR #54442 for the same grammar rejected tokens / Failed to advance FSM ... Terminating request kills on structural-tag tool calls under DSpark (v41/0018).

Reported
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub, posted by @schakaqt, 2026-09-02
Source: repo github.com/Zanooda/deepseek-v4.1-flash-cmp170hx by Zanooda on GitHub

Schaka's patch series 0007 to 0023

What the series is and how it is built

Schaka/deepseek-v4-cmp170hx is a copy of allover326's recipe (patches 0001 to 0006 plus 0005a and RESULTS.md byte-identical, so its headline numbers are allover326's 4-card measurements) plus 16 patches added from 2026-08-20 to 09-01 against haosdent/vllm base c3046d1. The range needs a full source build with the repo's docker/Dockerfile.fullbuild; how to obtain c3046d1 is on DeepSeek V4 Flash 0731 on 4x 8 GB (allover326 PP4 DSpark). The README is stale: it says 16 patches and its quick start checks out f8ea5bb, while 0007 onward target c3046d1. The authoritative launcher is launch/run-pp-dspark-podman.sh:

--pipeline-parallel-size 4 --gpu-memory-utilization 0.85 --max-model-len 750000 --max-num-batched-tokens 2048 --max-num-seqs 8 --kv-cache-dtype fp8 --block-size 256 --tokenizer-mode deepseek_v4 --enable-auto-tool-choice --tool-call-parser deepseek_v4 --reasoning-parser deepseek_v4 --override-generation-config '{"top_p":0.95}'
# environment
DSV4_LOGITS_ROW_CHUNK=64 DSV4_REP_FUZZY_MIN_HITS=15 DSV4_RECOVERY_MODE=nudge

On 3 cards with the INT4 repack it ran max-model-len 524,288 with a 2,013,978-token pool at about 86 tok/s.

Reported 8 GB
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub
Source: @schakaqt, 2026-09-02

Patch by patch

Apply in glob order on top of 0002 to 0006 and 0005a.

Patch	What it does	Origin
0007	NaN to `-inf` guard before `tl.argmax` in the rejection sampler; stops DSML and tool-call corruption under DSpark on sm_80 with fp8 KV	port of upstream vLLM #50183
0008	0731 thinking and effort defaults (an omitted `thinking` no longer renders a chat-mode prompt), parser mode synced with the template, argmax clamped to the vocabulary size	adapted from haosdent/vllm#21 (draft)
0009	DSML orphan-invoke recovery: a tool call starts from a bare invoke marker when the opening wrapper is missing	port of vLLM PR #49117 (fixes issue #48931)
0010	no recovery when the request declares no tools	rasatpetabit's gist
0011	a recovered call must close before it commits	rasatpetabit's gist
0012	a recovered call must carry its tool's required arguments	rasatpetabit's gist
0013	fixes the 0012 argument check accepting everything on configs without an argument converter	rasatpetabit's gist
0014	a malformed `required` schema field no longer raises an error	rasatpetabit's gist
0015	very deep argument JSON no longer crashes the check; `required` must be a list of strings	rasatpetabit's gist
0016	the same recovery while the parser is still in the reasoning state	Schaka
0017	retired; replaced by 0020 and 0021	Schaka
0018	accepts the `<｜DSML｜_tool_calls>` wrapper variant	ported from ChinaBoy0618/deepseek-v4-cmp170hx
0019	grammar-constrained decoding failures under DSpark no longer kill the request or worker	ported from ChinaBoy0618/deepseek-v4-cmp170hx
0020	control-tag soup and repeated-line tripwires, plus a fuzzy varied-wording detector	ChinaBoy0618, fuzzy detector by Schaka
0021	when a tripwire fires, splice a nudge and continue the same turn	Schaka
0022	opt-in `DSV4_RECOVERY_MODE=reopen`: splice a real turn boundary instead of free text (untested when published)	Schaka
0023	also emit `reasoning_content` beside `reasoning`	Schaka

Reported
Source: repo github.com/Schaka/deepseek-v4-cmp170hx by Schaka on GitHub, posted by @schakaqt, 2026-09-02

DeepSeek V4 Flash 0731 on 4x 8 GB (allover326 PP4 DSpark), DeepSeek V4 Flash 0731 on 3x 8 GB (PP3 DSpark), DeepSeek V4.1 Flash on 4x 8 GB (GNmac), DeepSeek V4.1 Flash on 4x 8 GB (Morrowmake), DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build), DeepSeek V4.1 Flash on 8x 8 GB (Zanooda PP8), vLLM on sm_80: known bugs and patches, Speculative decoding on the 170HX, Parallelism over the PCIe link, Engines, Memory errors and ECC, Xid errors, Instability under load, Troubleshooting.