Qwen3.8 Flash Next on 2x 8 GB (Qwen-Flash-SM80, GDS TEP2)

Qwen3.8-Flash-Next (nvidia/Qwen3.8-Flash-Next-NVFP4, revision fc694b54, 11 shards, 132.68 GB)
Reported 2 x 8 GB
Quant	NVFP4 main weights through Marlin (no native FP4 on sm_80), mixed FP8 and BF16 MTP weights, FP8 PLE table (about 51.2 GB) kept on NVMe
Engine	Qwen-Flash-SM80 overlay on vllm/vllm-openai:qwen38-flash-next (digest-pinned), FlashInfer PCIe IPC AllReduce, GDS/cuFile over NVMe P2PDMA v0.3.0 (commit 4d229f1, 2026-10-14); base vLLM 0.1.dev20073+g8e685d198
Topology	TEP2 (TP2 plus expert parallel), MTP 6; one card at PCIe Gen2 x16 (x16 mod), one at Gen2 x4; GPU P2P required
Context	262,144
Decode	168.29 tok/s
Prefill	1986.38 tok/s
VRAM per card	64 GB, --gpu-memory-utilization 0.93 (v0.3.0)
Driver	NVIDIA open 610.43.03, CUDA 13.0
Unlock build	bayley/cmpunlocker 5a7bb4b, sudo ./install.sh --p2p --no-iommu
Flags	`--tensor-parallel-size 2 --enable-expert-parallel --quantization modelopt --linear-backend marlin --moe-backend marlin --dtype bfloat16 --max-model-len 262144 --block-size 3456 --enable-prefix-caching --mamba-cache-mode align --enable-chunked-prefill --max-num-batched-tokens 8192 --no-async-scheduling, MTP 6 (greedy draft, standard rejection); env VLLM_PLE_GDS=1 CUFILE_USE_PCIP2PDMA=1; nvidia module RMForceStaticBar1=1;RMPcieP2PType=1`
Tested	@nguyenthimy2022kg-alt on 2026-09-08
Source	link
Files	config/vllm-args.json, config/runtime-env.json, config/example.json, scripts/serve.py and the data preparation scripts (Apache-2.0); about 24 MB of precompiled Triton kernels for the reference stack (in the repository, not mirrored here)
Confirmations	None yet.

Qwen-Flash-SM80 is a vLLM runtime for Qwen3.8-Flash-Next in NVFP4 on two 64 GB cards. It runs tensor plus expert parallel across the pair (TEP2) and MTP with 6 draft tokens. The 51.2 GB FP8 PLE lookup table (the n-gram table) stays on an NVMe SSD: one GPU reads the rows it needs straight from the SSD with GPUDirect Storage (cuFile over NVMe P2PDMA) and passes them to the other. ti posted it on Discord on 2026-09-09 at "200 token/s", and clee pointed two-card owners to it on 2026-10-01. The infobox shows the 2026-09-08 single-request run at 8K input; the current release, v0.3.0, reports 749.67 tok/s aggregate at 10 concurrent requests. Every number on this page is the repository's own reading from one machine, and nobody has posted a reproduction yet. The recipe needs working GPU P2P and a validated GDS path and will not start without P2P.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09
Source: @c133, 2026-10-01

Prerequisites

Driver: bayley/cmpunlocker at 5a7bb4b, installed with --p2p --no-iommu

For BAR1 and P2P the repository sends CMP owners to bayley/cmpunlocker at commit 5a7bb4b. That revision already contains the BAR1 and P2P driver patches the repository used to ship, so do not apply the repository's old patches on top (its drivers/cmp-bar1 folder now only keeps old links). Install with sudo ./install.sh --p2p --no-iommu: --p2p compiles in the P2P capability override, and --no-iommu stops the installer from adding iommu=pt. --no-iommu only skips that change; it does not turn off an IOMMU that is already on, and the installer does not set up static BAR1. Use matching NVIDIA driver libraries, firmware and kernel headers, then cold boot. A host with working P2P and GDS can skip this. Background: Driver unlock, Cmpunlocker forks, P2P.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09
Source: repo github.com/bayley/cmpunlocker by bayley on GitHub

P2P reported OK is not P2P working: check with a real copy

bayley's README warns that --p2p forces the advertised capabilities to OK, so canAccessPeer and nvidia-smi topo -p2p report success whether or not a single byte moves. Verify with an actual copy, ideally one that seeds the destination with a pattern that appears nowhere on the source. bayley measured 1.68 GB/s each way on PCIe Gen2 x4 over BAR1 P2P. The Qwen-Flash guide asks for nvidia-smi topo -p2p r and topo -p2p w to read OK, then real CUDA peer copies (for example the CUDA Samples p2pBandwidthLatencyTest). See Verifying an unlock and the failure case under Known issues.

Reported 8 GB
Source: repo github.com/bayley/cmpunlocker by bayley on GitHub
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub

Module options, kernel command line and BAR1 on the reference host

The reference host keeps these nvidia options in one file under /etc/modprobe.d; if an NVreg_RegistryDwords line already exists, merge into it instead of adding a second one:

options nvidia NVreg_EnableResizableBar=1 NVreg_RegistryDwords="RMForceStaticBar1=1;RMPcieP2PType=1;RmForceDisableIomapWC=1;RmForceEnableGen2=1;RMPcieLinkSpeed=0x1"

The last two entries are that host's PCIe Gen2 settings (Gen2 link retrain), not GDS requirements. Never add ForceP2P=0: it conflicts with this BAR1 path. The rest of the host state:

Kernel command line amd_iommu=off iommu=off (Intel and virtualised hosts need their own assessment; turning the IOMMU off also ends device passthrough and DMA isolation, so this is a bare-metal setup).
options nvme_core multipath=N, only if no real multipath namespaces are in use.
The filesystem that holds the PLE data mounted ext4 with data=ordered (rootflags=data=ordered for the root filesystem).
A 64 GiB BAR1 on each card: check the total under "BAR1 Memory Usage" in nvidia-smi -q -d MEMORY. If BAR1 stays small, fix BIOS address allocation or driver resizing first; a forced capability flag does not replace it (Resizable BAR and BAR1 64 GB).

Then run update-initramfs -u and update-grub and cold boot. More on these options: Kernel parameters and module options.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

Apply module options by reboot, never by reloading the driver

nvidia loads from the initramfs, so a new file in /etc/modprobe.d does nothing until update-initramfs -u and a reboot. bayley lists two ways to hang the machine hard enough to need a power cycle: NVreg_RegistryDwords="ForceP2P=0x11", and unloading and re-inserting the nvidia module on a live system.

Reported 8 GB
Source: repo github.com/bayley/cmpunlocker by bayley on GitHub

Kernel built with CONFIG_PCI_P2PDMA=y and CONFIG_ZONE_DEVICE=y

The SSD to GPU path uses Linux NVMe P2PDMA, without the older nvidia-fs module or custom NVMe patches. The kernel must have CONFIG_PCI_P2PDMA=y and CONFIG_ZONE_DEVICE=y; the version number alone does not tell you (grep -E '^CONFIG_(PCI_P2PDMA|ZONE_DEVICE)=' /boot/config-$(uname -r)). The reference host runs Ubuntu 24.04.4 with kernel 6.17.0-23-generic, driver 610.43.03, CUDA 13.0 and the cuFile and GDS tool packages libcufile-13-0 and gds-tools-13-0 at version 1.15.1.6-1. Simulate the apt install with -s first and make sure it does not pull a driver metapackage that replaces the adapted CMP driver. For the older nvidia-fs route see Gds-nvme-patch.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

GDS acceptance: gdscheck, then a verified 64 MiB read on the PLE drive

GPU P2P and SSD direct reads are separate checks; passing one proves nothing about the other. With a strict cuFile configuration (use_pci_p2pdma on, compatibility mode allowed and forced both off, CUFILE_USE_PCIP2PDMA=1, CUFILE_ALLOW_COMPAT_MODE=0), gdscheck.py -p must show "NVMe P2PDMA: Supported", use_compat_mode and force_compat_mode false, and IOMMU disabled. A line "NVMe: Unsupported" next to it refers to the nvidia-fs path and is not a failure. Then write 64 MiB with gdsio -V to a directory on the NVMe that will hold the PLE data (not /tmp or tmpfs) and read it back:

/usr/local/cuda-13.0/gds/tools/gdscheck.py -p
gdsio -f "$TEST_DIR/gds-test-64m.bin" -d 0 -m 0 -w 1 -s 64M -o 0 -i 4M -x 0 -I 1 -V   # write
gdsio -f "$TEST_DIR/gds-test-64m.bin" -d 0 -m 0 -w 1 -s 64M -o 0 -i 4M -x 0 -I 0 -V   # read and verify

A pass is DataSetSize: 65536/65536(KiB) and ops: 16 with no short reads or verification errors, plus 16 "cuFile p2p chunk read" operations of 4 MiB with errno: 0, p2p mode: 1, compat: 0 and bounce-buffer ptr 0 in the cuFile log at TRACE level, and the GPU bus id in that log matching the reading GPU. An exit code of 0 is not proof. The reference host read 4 GiB sequentially at 2.562 GiB/s (a historical run, not a guarantee for other drives).

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

Disk, host RAM and which GPU reads the table

Linux x86_64, Python 3.10 or newer, Docker with the NVIDIA Container Toolkit, a CUDA 13 compatible driver.
Disk: the checkpoint is 132.68 GB, the GDS conversion adds about 51.2 GB and the loading view 2.52 GB (hard links on the same filesystem). Reserve at least 220 GB plus Docker images; across filesystems the view copies the weights and needs about 80 GB more.
Host RAM: the reference host has about 32 GiB plus 8 GiB swap. The container is capped at 21 GiB RAM and 26 GiB RAM plus swap, and the launcher stops the service when available host RAM falls below 2 GiB.
The first entry of gpu_ids in config/local.json is the reading GPU: it pulls the PLE rows for each step from the SSD and broadcasts them over NCCL to the other card. On the reference host that is the Gen2 x16 card, which sits on the same CPU as the SSD on a different root port, with no PCIe switch anywhere on the path; the pair shows NODE in nvidia-smi topo -m. Pick the GPU that passes the read check above.
A PCIe switch board was not needed on that host, which is no guarantee for another platform (PCIe switch boards (PEX88096, PEX88064, PEX8747, PLX8749)).

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

Steps

Get GPU P2P and the GDS read check passing (Prerequisites).
Clone nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX and download nvidia/Qwen3.8-Flash-Next-NVFP4 at revision fc694b54fb0174e0913e6adf86691ef85a4ead47 (11 shards, 132.68 GB) onto the NVMe filesystem that passed the GDS check. The guide uses huggingface_hub==1.29.0; re-running the download resumes it.
Run scripts/check-model.py (pinned metadata hashes, tensor headers and lengths), then scripts/prepare-nvidia-view.py, which builds a loading view without the PLE rows so model loading does not scan the whole table. The original checkpoint stays as it is.
Convert the PLE table to the GDS layout from the original checkpoint index (not the view): prepare-ple.py mapping, convert-ple-to-gds-layout.py --compact --block-rows 256, then prepare-ple.py enroll, which verifies the output and writes config/local-ple-identity.json. The conversion streams the table and never loads all 51.2 GB into RAM.
Build the image with docker build -t qwen-flash-sm80:0.3.0 . The build does not need GPU access.
Copy config/example.json to config/local.json; set image, models_root, model_subdir (the view), ple_artifact, max_num_seqs 10 and gpu_memory_utilization 0.93, and review gpu_ids, the paths and port.
Start with python3 scripts/serve.py start --config config/local.json --dry-run to print the command, then without --dry-run. Wait for loading and warmup until /health returns HTTP 200; a submitted start is not a ready service. Logs go to runs/<container>/.
Connect an OpenAI-compatible client to http://127.0.0.1:18420/v1 with model qwen3.8-flash-next and a placeholder API key. The service listens on localhost without authentication; for another machine, forward the port over SSH (ssh -N -L 18420:127.0.0.1:18420 user@server).

git clone https://github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX.git
cd Qwen-Flash-SM80-170HX
hf download nvidia/Qwen3.8-Flash-Next-NVFP4 --revision fc694b54fb0174e0913e6adf86691ef85a4ead47 \
  --local-dir "$DATA/models/NVIDIA-Qwen3.8-Flash-Next-NVFP4"
# check, build the loading view, convert and enroll the PLE table (step 3 and 4), then:
docker build -t qwen-flash-sm80:0.3.0 .
python3 scripts/serve.py start --config config/local.json --dry-run
python3 scripts/serve.py start --config config/local.json
curl --fail http://127.0.0.1:18420/health

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

The image: a digest-pinned vLLM build plus the overlay

The Dockerfile starts from vllm/vllm-openai:qwen38-flash-next@sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8 (vLLM 0.1.dev20073+g8e685d198), adds libcufile-13-0 at version 1.15.1.6-1, installs pinned kernel packages by hash (TileLang 0.1.14), compiles the GDS reader (csrc/) and the PCIe communication module, and applies the runtime overlay (src/) with FLASHINFER_CUDA_ARCH_LIST=8.0. It deliberately does not upgrade PyTorch, Triton or NumPy inside the base image, and patches/ holds upstream source hashes so the overlay refuses to overwrite an incompatible version. To upgrade, rebuild the image.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

Launch arguments and environment

config/vllm-args.json holds the server arguments; the main ones:

--tensor-parallel-size 2 --pipeline-parallel-size 1 --enable-expert-parallel
--quantization modelopt --linear-backend marlin --moe-backend marlin --dtype bfloat16
--max-model-len 262144 --block-size 3456 --enable-prefix-caching --prefix-match-unit 36
--mamba-cache-mode align --enable-chunked-prefill --max-num-batched-tokens 8192 --no-async-scheduling
--speculative-config '{"method": "mtp", "num_speculative_tokens": 6, "draft_sample_method": "greedy", "rejection_sample_method": "standard"}'
--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder
--limit-mm-per-prompt '{"image": 128, "video": 0}'
# config/runtime-env.json (selection)
VLLM_PLE_GDS=1 CUFILE_USE_PCIP2PDMA=1 VLLM_PLE_GDS_WORKERS=32 VLLM_PLE_GDS_STAGING_SLOTS=3
Q38_PCIE_IPC=1 FLASHINFER_CUDA_ARCH_LIST=8.0

max_num_seqs in config/local.json takes 1 to 10 and defaults to 1 when left out. The 262,144-token limit is per request; requests share one VRAM cache rather than each getting a full context. Prefix matching works in 36-token units while physical cache blocks stay at 3,456 tokens. Keep synchronous scheduling. Defaults: thinking on with preserved reasoning history at reasoning_effort=xhigh, temperature 1, top-p 0.95, top-k 20; for timing runs the guide sends temperature 0 with thinking off. Up to 128 images per request are configurable (a limit, not a promise that 128 fit), and video is off.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

Measured numbers

v0.3.0: 749.67 tok/s aggregate at 10 requests, 76.29 tok/s per request

The repository's 10-request run (2026-10-11): NVIDIA NVFP4 checkpoint, TEP2 plus MTP6, the prompt "Build a web page" (sent in Chinese), thinking off, temperature 1, top-p 0.95, top-k 20, 10 requests of 5,000 output tokens each with the end-of-sequence token ignored. Output speed runs from the first to the last non-empty streamed output and leaves out time to first token. Per-request speed ranged 75.19-78.30 tok/s. The three stages of the optimisation series:

Stage	Mean tok/s per request	Aggregate tok/s	Generation cycle
Before the series	63.43	623.41	87.73 ms
Compact vocabulary transport	71.54	700.98	76.88 ms
Multi-request PCIe transport (v0.3.0)	76.29	749.67	72.34 ms

Each setting had one screening run. The release checks covered short text, JSON and request lifecycle; ten full-context requests, concurrent image requests and long stress runs were not rerun on this release.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

Single request, 2026-09-08: 151 to 168 tok/s decode and about 2,000 tok/s prefill from 8K to 128K

An earlier release on the RadixArk NVFP4 checkpoint, thinking on at the template's default xhigh, TEP2 plus MTP6, output limit 50,000 tokens, one run per length (reasoning and answer tokens both count as output):

Input	Prefill tok/s	Decode tok/s
8K	1986.38	168.29
16K	2083.10	152.99
32K	2068.85	156.44
64K	2036.21	159.88
128K	2059.93	151.05

Prefill here is a client-side estimate: input tokens divided by the time to the first non-empty output. A temperature-0 run reached 170.83 tok/s with MTP acceptance 0.576, and outputs with less reasoning came close to 200 tok/s (199 shown by the client). The author calls the 259 to 260 tok/s figure from a counting test misleading. ti put the prefill cap near 2,000 tok/s down to the PCIe 2.0 x4 link. GalyGames reposted this table on 2026-09-13 and said it was not his work, so it is not a second run.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09
Source: @galygious, 2026-09-09

From about 38 to 170.83 tok/s over the project's life

The repository puts the first deployment at about 38 tok/s and the best single-request result at 170.826 tok/s, 4.5x the start. These are results from different stages of the project, not a controlled comparison with stock vLLM.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub

Known issues

Will not start without P2P, and topo -p2p OK is not enough

fdm (2026-09-19) unlocked two 8 GB cards with bayley/cmpunlocker at 5a7bb4b, installed with --p2p, on driver 610.43.02. nvidia-smi topo -p2p rw returned OK and ACS was off, but a real GPU0 to GPU1 copy ran at 1.71 GB/s and the target buffer stayed empty. His cards were wired through MCIO on a Supermicro board, on AMD root ports with no PCIe switch and no x16 mod. He needed P2P for this deployment, which will not start without it, and asked whether BAR1 P2P works directly on AMD root ports. The reference host also runs its cards on CPU root ports with no switch (Prerequisites); what made the difference in his case was not settled in the thread. Check P2P with a content-checked copy before you download 132.68 GB (P2P).

Reported 8 GB
Source: @filth5585, 2026-09-19
Source: @napsterae, 2026-09-20

Validated on one machine

The recipe has been validated only on the author's host: a Huananzhi H12D-8D V2.0 (Huananzhi H12D-8D) with one AMD EPYC 7532 (32 cores, one NUMA node), about 32 GiB RAM and 8 GiB swap, a Samsung 990 PRO 4TB at PCIe 4.0 x4 as the PLE drive, one card at Gen2 x16 (PCIe x16 capacitor mod) and one at Gen2 x4, Ubuntu 24.04, kernel 6.17, NVIDIA open driver 610.43.03 and CUDA 13.0. Other GPUs have not completed a full-model run, and a fresh GDS setup with the public driver revision has not been validated on a second machine.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

Precompiled Triton kernels tied to the reference stack

src/preload/ and src/memory_extra_preload/ hold about 24 MB of preloaded Triton kernels with SHA256 manifests, built for this fixed SM80 environment (they contain no model weights). They belong to the pinned image and the reference stack: keep the digest-pinned base image and do not upgrade PyTorch or Triton inside it. Get them from the repository; this wiki does not mirror them.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

Do not pair RadixArk PLE data with NVIDIA weights

v0.3.0 uses the NVIDIA checkpoint. A v0.1.7 deployment on the RadixArk checkpoint has to redo the download, the loading view and the PLE conversion (Steps 2 to 4) and update the model, PLE and identity paths; keep the old image and configuration for rollback. NVIDIA model and PLE data prepared for v0.2.0 can be reused. When upgrading, set image to the new build, keep draft_int8=true and set max_num_seqs 10 and gpu_memory_utilization 0.93.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

If direct reads fail: the host-read path, and a RAM disk only with about 73 GiB free

A later build on main adds an optional path (Q38_PLE_HOST_READ=1, rebuilt image) that reads PLE rows into a small pinned host buffer and copies them to the GPU, meant for investigating direct-read failures on another host. It still needs cuFile to initialise and register the table, so it does not rescue a host where cuFile fails, and it is not a confirmed fix for every hang. The table can also sit on a RAM disk (Linux brd with ext4, not tmpfs), which needs about 73 GiB of free RAM (47.7 GiB of payload plus a 24 GiB reserve); a 32 GiB host stays on the SSD path. If the service stops with memory-stop.txt, the launcher's 2 GiB free-RAM guard tripped: check other workloads before restarting.

Reported 8 GB
Source: repo github.com/nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX by nguyenthimy2022kg-alt on GitHub, posted by @jovial_dolphin_29796, 2026-09-09

Untested: caching PLE rows for about 20% more decode

mrpandagod suggested, without testing it, that caching PLE rows could add about 20% decode and 30% prefill and spare the SSD. The v0.3.0 runtime environment sets VLLM_PLE_GDS_CACHE_ROWS=0; nobody has posted numbers either way.

Theory 8 GB
Source: @djpandaman, 2026-09-09

Qwen3.8 Flash Next on 2x 8 GB (ElectroHobo W4A16 TP2) · Qwen3.8 Flash Next on 2x 8 GB (Plean PP2) · Qwen3.8 Flash Next on 2x 8 GB (DUFRelic PP2) · Qwen3.8 Flash Next on 2x 8 GB (Morrowmake EXL3) · Qwen3.8 Flash Next on 1x 8 GB (iIIusi0n SSD PLE) · Qwen3.8 Flash Next on 3x 8 GB (clee PP3) · Qwen3.8 Flash Next on 4x 8 GB · P2P · Gds-nvme-patch · Huananzhi H12D-8D · Kernel parameters and module options · Resizable BAR and BAR1 64 GB · PCIe x16 capacitor mod · Parallelism over the PCIe link · Speculative decoding on the 170HX · vLLM on sm_80: known bugs and patches · Quantization on the 170HX · Engines · Benchmarks

Sources

Repository: nguyenthimy2022kg-alt/Qwen-Flash-SM80-170HX (Apache-2.0; v0.3.0, commit 4d229f1, last commit 2026-10-14): English README, deployment guide, CMP P2P guide, NVMe GDS guide, reference hardware and multi-request records. The GitHub account is probably ti's (@jovial_dolphin_29796), but that link is not confirmed.
Driver: bayley/cmpunlocker at commit 5a7bb4b (README sections on IOMMU, verification and GPU-to-GPU P2P).
The 2026-09-09 thread where ti posted the repository, with GalyGames, DeeJay and mrpandagod: Source: @jovial_dolphin_29796, 2026-09-09
Source: @galygious, 2026-09-09
Source: @deejay9983, 2026-09-09
Source: @djpandaman, 2026-09-09
clee's pointer for two-card owners: Source: @c133, 2026-10-01

Prerequisites

Driver: bayley/cmpunlocker at 5a7bb4b, installed with --p2p --no-iommu

P2P reported OK is not P2P working: check with a real copy

Module options, kernel command line and BAR1 on the reference host

Apply module options by reboot, never by reloading the driver

Kernel built with CONFIG_PCI_P2PDMA=y and CONFIG_ZONE_DEVICE=y

GDS acceptance: gdscheck, then a verified 64 MiB read on the PLE drive

Disk, host RAM and which GPU reads the table

Steps

The image: a digest-pinned vLLM build plus the overlay

Launch arguments and environment

Measured numbers

v0.3.0: 749.67 tok/s aggregate at 10 requests, 76.29 tok/s per request

Single request, 2026-09-08: 151 to 168 tok/s decode and about 2,000 tok/s prefill from 8K to 128K

From about 38 to 170.83 tok/s over the project's life

Known issues

Will not start without P2P, and topo -p2p OK is not enough

Validated on one machine

Precompiled Triton kernels tied to the reference stack

Do not pair RadixArk PLE data with NVIDIA weights

If direct reads fail: the host-read path, and a RAM disk only with about 73 GiB free

Untested: caching PLE rows for about 20% more decode

Related

Sources