MiMo V2.6 Flash on 4x 8 GB

MiMo-V2.6-Flash-RL (309B MoE, 15B active; text, image, video and audio input)
Reported 4 x 8 GB
Quant	NVFP4 (ModelOpt mixed precision): ProCreations/MiMo-V2.6-Flash-RL-NVFP4, run as 4-bit weights with BF16 activations through Marlin
Engine	Morrowmake/vllm-cmp170hx image (tag qwen38-recipe-1.0.0) plus one mounted file (vLLM #58235 vision-sink fix) recipe 0.2.1; engine 18305948ac
Topology	TP4 (P2P on); expert parallel off
Context	1,048,576
Decode	274.9 tok/s
Prefill	2722 tok/s
VRAM per card	64 GiB; weights 45.5 GiB plus 10 GiB KV per card; GPU_UTIL 0.94 (fixed)
Power cap	180 W
Driver	580 or newer (measured on 610.57.04)
Unlock build	cmpunlocker (64 GiB, 74 SMs) with a P2P-capable build
Flags	`--tensor-parallel-size 4 --kv-cache-dtype bfloat16 --block-size 128 --max-model-len 1048576 --gpu-memory-utilization 0.94 --kv-cache-memory 10737418240 --max-num-seqs 32 --max-num-batched-tokens 2048; DFlash 7/5/3 by load`
Tested	@nathang6866 on 2026-10-18
Source	link
Confirmations	None yet.

Xiaomi's MiMo-V2.6-Flash-RL on four 64 GB cards: the ProCreations NVFP4 checkpoint, tensor parallel 4, the full 1M-token context, MiMo's own DFlash drafter, reasoning and tool calls, with image input. It runs Morrowmake's engine image as released apart from one mounted vision fix. Scripts descend from Morrowmake's Qwen recipe and the launch configuration from kaka86mm/vllm-mimo-170hx, the first MiMo-V2.6 deployment on these cards. Infobox numbers are one user on code. Source: repo github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-18
Source: @nathang6866, 2026-10-18

Prerequisites

Four CMP 170HX exposing 65,536 MiB with 74 SMs, PCIe Gen2 x16 (the x16 capacitor mod), peer-to-peer on (the author's four sit on one PLX switch). x4 links are not measured. A card exposing less than 64 GiB will not fit 45.5 GiB of weights plus 10 GiB of KV. See Driver unlock, Unlocks.
Driver 580 or newer, Docker with the NVIDIA Container Toolkit, about 190 GiB of free disk plus a 20 GB engine image. Author's host: 2x Xeon E5-2697A v4, 204 GiB RAM.

Source: repo github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-18

Steps

git clone https://github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe
cd mimo-v26-flash-cmp170hx-recipe
./start.sh          # pull the engine image, fetch the model, start, wait for /health

Checkpoint ProCreations/MiMo-V2.6-Flash-RL-NVFP4 at 34cebc5d, 183 GiB (its gguf/ folder is skipped); the DFlash drafter is the checkpoint's dflash/ folder.
Image ghcr.io/morrowmake/vllm-cmp170hx@sha256:5b45c4d6feafefe2eba962375b0a9a0b64fd22b9e05abb4d1966f14a32a2db79.
Default command: --hf-overrides '{"architectures":["MiMoV2OmniForCausalLM"]}' --trust-remote-code --tensor-parallel-size 4 --kv-cache-dtype bfloat16 --block-size 128 --max-model-len 1048576 --gpu-memory-utilization 0.94 --kv-cache-memory 10737418240 --max-num-seqs 32 --max-num-batched-tokens 2048 --reasoning-parser mimo --enable-auto-tool-choice --tool-call-parser mimo --enable-prefix-caching --async-scheduling --prefill-chunk-with-decodes 384 --max-num-partial-prefills 2 plus --speculative-config '{"method":"dflash","model":"<ckpt>/dflash","num_speculative_tokens":7,"num_speculative_tokens_per_batch_size":[[1,1,7],[2,2,5],[3,32,3]]}'.
Env: VLLM_CUSTOM_ALLREDUCE_FLAGS=1 VLLM_WORKER_MULTIPROC_METHOD=spawn PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False CUDA_DEVICE_ORDER=PCI_BUS_ID, plus VLLM_CUSTOM_ALLREDUCE_PCIE_P2P=1 and NCCL_P2P_LEVEL=SYS after the P2P check.
0.2.1 adds PYTHONPYCACHEPREFIX=/cache/pycache: start-to-/health 410 s before, 302 s after.

Source: repo github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-18

Measured numbers

2026-10-18, MiaAI-Lab protocol (400-token replies, temperature 0, thinking off). Tok/s structured / code / prose:

	DFlash (default)	MTP (`SPEC_MODE=mtp`)	`P2P=off`
1 user	322.0 / 274.9 / 170.1	128.9 / 119.6 / 114.5	181.3 / 164.4 / 96.8
2 users	427.0 / 397.7 / 255.1
4 users	572.7 / 538.3 / 412.1
8 users	753.8 / 738.0 / 555.5	417.2 / 379.8 / 362.3	426.1 / 388.9 / 254.3

Cold prefill 2,722 tok/s (24K to 38K prompts); KV pool 1,615,953 tokens (1.54 full-length requests); with MTP and FP8 KV the pool grows to 1,823,860.
HumanEval 161/164, GSM8K 195/200 (first 200, thinking on). Decode step 15 ms at 1 user, 21 ms at 2, about 48 ms at 8.
DFlash acceptance about 99% on counting, about 50% on code, about 20% on prose. The draft schedule (7 for 1 request, 5 for 2, 3 from 3) gave +11 / +12 / +39% at 8 users over a fixed 7.
P2P off is about 40% slower on every row, so a P2P-capable driver is essential for TP4 here. Expert parallel was 4 to 9% slower, so it is off.
Soak (0.1.0): 20 minutes, 8 concurrent streams plus one image request every 30 s: 1,646 requests, 0 failures, a steady 443 tok/s, 40 of 40 images correct, no Xid, cards at 41 to 43 C afterwards.

Source: repo github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-18

Discord update on 2026-10-18: 322 / 275 / 170 tok/s single user, 754 / 738 / 556 at 8, 2,720 tok/s prefill, 1.62M-token KV (against 1.2M for GLM Flash TP4). ElectroHobo expects a DFlash2 drafter for MiMo would close the speed gap to GLM Flash. Source: @nathang6866, 2026-10-18
Source: @nathang6866, 2026-10-19

Is it better than GLM 5.3 Flash?

Contested

Miséricorde: not better than GLM, and less KV. Source: @wakeupmbrc1, 2026-10-19
ElectroHobo: 1.6M KV against 1.2M for GLM Flash TP4. Source: @nathang6866, 2026-10-19
segmond: on his own test MiMo 2.6 Flash crushed DeepSeek V4.1 Flash 1017, GLM 5.3 Flash and Qwen 3.8 Flash, with output close to what Opus 5.5 produced. Source: @segmond, 2026-10-25

Known issues

Image input needs the mounted patches/mimo_v2_omni.py (upstream vLLM #58235: the vision attention sink as an extra softmax logit). Without it a red/blue split image is described as "white and red" or "entirely red" (VISION_PATCH=0 reproduces that); text output is identical either way. Video and audio input are untested.
FP8 KV is only possible with MTP or no drafter: the DFlash drafter runs on the Triton attention backend, which has no FP8 KV path on sm_80.
KV_OFFLOAD_DIR is write-through and never pruned. FAIR_CHUNK=384 keeps a mid-answer user at about 15 tok/s while a 30K prompt lands, at the cost of 20% slower prompt reading.
Community recipe, not endorsed by Morrowmake; one machine; the vision patch is pinned to engine commit 18305948ac.

Source: repo github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe by ElectroHobo on GitHub, posted by @nathang6866, 2026-10-18

An earlier PP4 recipe with the official FP8 weights is on localairecipes ("MiMo-V2.6-Flash-RL omni on 4x CMP 170HX PP4 official fp8"); Mountains' own first attempt at MiMo was badly broken. Nobody in the channel had tried MiMo 2.6 Pro, which needs more cards. That recipe is kaka86mm's: PP4 with the official FP8 weights (161 GB), no speculative decoding and an LMCache three-tier KV cache, at 64.5 tok/s for one user, the slow alternative to this page. Source: @mountains1605, 2026-10-07
Source: @darkmaniac7, 2026-10-27 (lab notes)
Source: @mountains1605, 2026-10-07

Sources

Repo: ElectroHobo/mimo-v26-flash-cmp170hx-recipe (MIT; engine Apache-2.0; checkpoint MIT), release 0.2.1 on 2026-10-28.

Prerequisites

Steps

Measured numbers

Is it better than GLM 5.3 Flash?

Known issues

Related

Sources