gds-nvme-patch
More actions
Reported 8 GB and 10 GB
gds-nvme-patch restores NVIDIA GPUDirect Storage (cuFile and nvidia-fs, direct NVMe-to-VRAM transfers) on Linux kernels where NVIDIA's own NVMe patch no longer applies. It patches the kernel's NVMe driver, not the GPU, and was built and tested on an unlocked CMP 170HX. Author: GitHub cachenetics, GPL-2.0. Repository: https://github.com/cachenetics/gds-nvme-patch Source: repo github.com/cachenetics/gds-nvme-patch by cachenetics on GitHub, posted by @mountains1605, 2026-10-20.
@markshowa asked whether GDS can run on the 170HX for model loading and KV cache; @mountains1605 pointed to this repository and said someone had done it a while ago. Source: @markshowa, 2026-10-20
Source: @mountains1605, 2026-10-20 Theory Expected gains: faster model loading without going through CPU and system RAM, and a place for n-gram tables, PLE and other low-bandwidth parts of inference. Neither poster had tried it. Source: @mountains1605, 2026-10-21
Source: @mountains1605, 2026-10-21
Scope
- Kernels 7.0 and 7.1 apply; 6.17 and 6.18 are refused; older kernels must use NVIDIA's MOFED/DOCA patch. x86_64 only. Needs kernel headers, the nvidia-open and nvidia-fs module sources, and CUDA with cuFile.
- Tested end to end on CachyOS with kernel 7.1.5. Kernel 7.0 (Ubuntu 26.04) is verified at patch level only. The Debian, Ubuntu and Fedora install hooks are written but were never run on real hardware.
- Mode A (rebuild and reboot, root on NVMe) is the only verified mode, and the installer refuses it without a second rescue kernel unless forced.
Source: repo github.com/cachenetics/gds-nvme-patch by cachenetics on GitHub, posted by @mountains1605, 2026-10-20.
Usage
git clone https://github.com/cachenetics/gds-nvme-patch && cd gds-nvme-patch
./probe.sh # read-only preflight, test-applies the patch to a copy
sudo ./install.sh --dry-run
sudo ./install.sh # --persist (default) adds a kernel-update rebuild hook
sudo ./uninstall.sh # restores the stock NVMe driver
Check with gdscheck.py -p (should show NVMe supported) and prove a transfer with gdsio. Installing the patch only makes GDS available: the loader must call cuFile, kvikio or DALI. For ext4 roots it adds rootflags=data=ordered to the kernel command line. Source: repo github.com/cachenetics/gds-nvme-patch by cachenetics on GitHub, posted by @mountains1605, 2026-10-20.
Known issue on CMP cards
Warning Loaders that read straight into many destination tensors (for example vLLM fastsafetensors) race NVIDIA's nvidia-fs bookkeeping and crash with a null dereference, or hang at 0 percent. The window is wider on slow PCIe links, so an unlocked 170HX at low PCIe width reproduces it readily. The fix is bounded staging, implemented in the companion loader cachenetics/vllm-gds-bounded (--load-format gds_bounded). The first GDS load after a boot is the most reliable; between loads run rmmod nvidia_fs && modprobe nvidia_fs or reboot. Source: repo github.com/cachenetics/gds-nvme-patch by cachenetics on GitHub, posted by @mountains1605, 2026-10-20.
Caveats
It modifies the root disk's NVMe driver. O_DIRECT reads with a small pinned buffer get most of the low-RAM benefit without any kernel patch. Last commit 2026-08-19, before newer kernels. Source: repo github.com/cachenetics/gds-nvme-patch by cachenetics on GitHub, posted by @mountains1605, 2026-10-20.