Virtual machines and passthrough: Difference between revisions

Latest revision as of 02:56, 1 November 2026

Reported 8 GB and 10 GB

Running CMP 170HX cards inside a virtual machine works, with more moving parts than bare metal. The short version:

Unlock where the NVIDIA driver runs. Under Proxmox/QEMU most people unlock on the host and pass the unlocked card in; in-guest unlocks also work on some stacks, and which route is better is contested (below). Under ESXi the unlock runs inside the guest.
64 GB VRAM works in a VM. It does not need a big BAR1.
Gen2 works in a VM, but how it is done differs per hypervisor, and the Gen2 path has caused guest hangs under Proxmox for some people. The guest cannot retrain the physical port by itself.
64 GB BAR1 in a VM is hard (ESXi needs a pre-boot step on every host boot).
P2P between CMPs inside a VM has not been shown to work. One Proxmox owner says it runs for him but posted no configuration or numbers; every verified CMP P2P result is on bare metal or a host kernel.
Bare metal is much simpler. Source: @darkmaniac7, 2026-10-29 (lab notes)

Where to run the unlock

Asked whether the unlock must happen on the hypervisor, benoit tried both on Proxmox: unlocking fully inside the VM gave him trouble unlocking both cards, while unlocking on the Proxmox host worked and is what he kept. Source: @jack0x01, 2026-10-08
Source: @bismuth67, 2026-10-08 SpiralGlide does the same: unlock on the host and pass the card into a container that runs Docker containers. Proven Source: @spiralglide, 2026-10-08

The PixelML club writes the same rule from its VFIO baseline: under VFIO the unlock can be applied host-side so the guest runs the stock driver; their second host showed Gen2 x16 and 64 GB in the guest that way. Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @djpandaman, 2026-10-24 (docs/INSTALLATION.md)

The cmpunlocker upstream "Virtual Passthrough" route also unlocks on the host, because under QEMU/VFIO the in-guest Booter path returns 0x15 (ACR_ERROR_FLCN_REG_ACCESS) and the guest ends up on a stock driver. Under ESXi that error did not appear and the whole v0.5 core ran inside the guest. Source: @darkmaniac7, 2026-10-21 (lab notes)

aidoc confirmed his register probe script can be run from inside a VM. Source: @sekkit_75019, 2026-10-04

Host or guest: not settled

Unlocking on the Proxmox host and passing the card through is the route most people keep, and the upstream passthrough route does the same. In-guest unlocks work on some stacks and failed on others. The reports, in the order they were posted:

Guest, worked. Capone had used the unlock inside a Proxmox VE VM by 2026-08-02. Source: @beefbonecapone, 2026-08-02
Guest, failed. On 2026-09-04 Bagster unlocked inside a Proxmox guest but got no Gen2, and saw the same problem as fafinet (then on build f51da03): many "restoring BAR" messages and a guest hang on reboot after running the unlocker in the guest. Bagster went back to unlocker v1 on that server. orrorin could not unlock in a guest VM at all. Source: @fafinet, 2026-09-04
Source: @bagstah, 2026-09-04
Source: @_orrorin, 2026-09-04
Guest, worked. SeriousRising runs Gen2 cards in separate Proxmox VMs, Polskie Kaktus runs the v1 unlocker inside a libvirt guest, and nightdays unlocks inside an ESXi guest (all below). Source: @seriousrising, 2026-09-06
Source: @polskiekaktus, 2026-09-07
Source: @.nightdays, 2026-09-07
Guest, broke after working. On 2026-09-18 Monkey55 reported that the unlocker, which worked in his Proxmox VM a month earlier, now makes the GPU disappear ("gpu not found"); he planned to open a ticket. Source: @amonkey55, 2026-09-18
Host, worked. benoit and SpiralGlide on 2026-10-08 (above): the in-guest unlock gave trouble with two cards, the host unlock worked. GalyGames: works on his Proxmox system (Debian based). Logy runs the cards in a VM under Proxmox and QEMU and points anyone with passthrough trouble to the anoane guide. ikyo runs six cards as 4 + 2 under Proxmox. Source: @bismuth67, 2026-10-08
Source: @galygious, 2026-10-10
Source: @logy0, 2026-10-10
Source: @ikyo, 2026-10-26
Guest, worked with fixes. Johnnydaboy's dual-CMP Proxmox fork (branch homelab-dual-cmp170hx, OVMF guest, 65,536 MiB and Gen2 x8 per card) and the lab's ESXi guest both unlock inside the guest. Gen2 inside a VFIO guest: the Consensus wiki and club-170hx say it does not train; the ESXi guest and Johnnydaboy's guest reach Gen2 x8 (in his case the physical root-port retrain runs on the Proxmox host). Source: @darkmaniac7, 2026-10-25 (lab notes)

Contested 8 GB and 10 GB

Reload a rebuilt driver with a VM power cycle

The reliable way to load a rebuilt CMP driver in a VM is a VM power cycle. In-guest rmmod/modprobe has wedged with D-state processes in uvm_va_space_destroy, and polyuij42 reports an unbind that hung in D state. vipic's VM compute hang was tied to bar1-resize-unlock.patch (see Problems reported under Proxmox).

Reported 8 GB and 10 GB
Source: @polyuij42, 2026-10-25
Source: @darkmaniac7, 2026-10-25 (lab notes)

Why run the CMP in a VM: driver pinning and mixed-GPU hosts

Eksitus suggested pinning the driver to one version, as P100 owners do; insomniac noted that breaks down when other GPUs in the box need newer drivers, and Zagitta suggested putting the CMPs in a passthrough VM (2026-09-07). Polskie Kaktus does exactly that, so his RTX 3080 can stay on the latest NVIDIA driver without waiting for the unlocker to keep up. SeriousRising separates AI workloads into VMs on servers with spare capacity, and DUFRelic cited easy backups. HumanDrone8721 found bare metal faster than any virtualization.

Reported
Source: @eksitus, 2026-09-07
Source: @ins_mniac, 2026-09-07
Source: @zagitta, 2026-09-07
Source: @humandrone8721_60165, 2026-09-08
Source: @dufrelic, 2026-09-08
Source: @polskiekaktus, 2026-09-08
Source: @seriousrising, 2026-09-08

Proxmox (QEMU/KVM, VFIO)

A working Proxmox host recipe

The most complete public Proxmox write-up is the anoane repo (MIT license), linked by Logy as the guide he followed: Source: repo github.com/anoane/01-proxmox-dual-gpu-passthrough by anoane on GitHub, posted by @logy0, 2026-10-10. It passes an RTX PRO 6000 Blackwell and an 8 GB CMP 170HX unlocked to 64 GB into one Ubuntu 24.04 guest. The author calls it a point-in-time transcription of one host, not a turnkey script. Key points from the repo (commit 865a608, 2026-09-22):

Host: Proxmox VE 9.2.x, kernel 7.0.x-pve, Secure Boot disabled (mokutil --sb-state must say disabled), AMD Ryzen 9 9950X3D. cmpunlocker pinned at commit 76f0954 on the host with driver 610.43.02. The guest runs Ubuntu 24.04.4, kernel 6.8.0-139-generic, the stock NVIDIA driver 610.43.02; the unlock lives on the host. The guest sees the CMP as "NVIDIA Graphics Device" with 65,536 MiB, so match it by PCI id 10de:20c2, not by name.
Host kernel command line: quiet amd_iommu=on iommu=pt; modules vfio vfio_iommu_type1 vfio_pci vfio_virqfd.
options vfio-pci disable_idle_d3=1: if vfio idles the CMP into D3, BAR0 reads all ones and the card disappears.
The CMP is armed and bound by cmpunlocker's boot service, not by the vfio-pci ids= list.
VM: machine q35 (required), bios seabios, cpu host, balloon 0, CMP passed as hostpci1: 0000:06:00.0,pcie=1,rombar=0. The CMP shared an IOMMU group with a PCIe bridge, so the whole group's function set is passed.
A pre-start hookscript re-runs cmpunlocker's gsp-restore for cards on vfio-pci before every VM start, because the vfio bind event only fires on a real driver change and later VM starts would otherwise skip the restore.
A VM that is killed rather than shut down leaves the GSP boot state latched and the next VM gets no GPU; only ./tools/passthrough.sh restore <bdf> clears it. cmp_no_bus_reset.ko blocks the bus reset vfio does at bind, which would otherwise erase the unlock.
PCIe: the card trains Gen1 at boot. A host-side gen2-hammer script walks the upstream bridge and retrains with setpci until LnkSta reads Gen2, run by a gen2.service before basic.target. The card was confirmed Gen2 x16 in a PCIe 5.0 x16 slot; in the documented build it sits in a PCIe 3.0 x1 slot and measured about 0.38 GB/s.

From the repo README and vm/ai-vm.conf: Source: repo github.com/anoane/01-proxmox-dual-gpu-passthrough by anoane on GitHub, posted by @logy0, 2026-10-10

More from the anoane repo

passthrough.sh prepare refuses a card that reports under 30,000 MiB, so a card whose unlock did not take is not passed through.
The GSP restore must only run on a card bound to vfio-pci: run while the host nvidia driver owns the card, it clobbers a live GSP. "GSP boot state already clean" in the hookscript output is normal.
The CMP trains Gen1 x1 at boot in that slot. gen2-hammer retries up to 600 times at 0.05 s intervals; gen2.service has TimeoutStartSec=45.
Every bandwidth figure in the repo is the x1 slot, not the card. Read LnkSta under load: an idle card shows downtrained speeds.

Reported 8 GB
Source: repo github.com/anoane/01-proxmox-dual-gpu-passthrough by anoane on GitHub, posted by @logy0, 2026-10-10

Where the anoane setup came from

anon33anon published this setup as anoane/01-proxmox-dual-gpu-passthrough on 2026-09-10, a month before Logy linked it, and said the latest unlocker version with Proxmox/VFIO Gen2 support works for him (driver 610.x; 615 not tried). The same day Dimon reported that after reinstalling Proxmox 9.1.1 his VM froze after the unlock. He got it running with the old unlocker for the main component and the retrain process on the host, after many steps (some wrong, by his account), and asked for clean from-scratch instructions.

Reported 8 GB
Source: repo github.com/anoane/01-proxmox-dual-gpu-passthrough by anoane on GitHub, posted by @anon33anon, 2026-09-10
Source: @dimon_91153, 2026-09-10

The upstream passthrough route (cmpunlocker master)

On 2026-09-07 Amogh announced VFIO support in cmpunlocker so Proxmox, KVM and QEMU would work (the passthrough branch), merged to master the same day:

Run ./install.sh on the host, not inside the VM.
Pass the unlocked card to the VM and use it there.

He named vGPU and MIG as the next targets.

Reported 8 GB and 10 GB
Source: @_j3ns, 2026-09-07

Why the upstream route unlocks on the host

The branch was merged to master as c40033b "Virtual Passthrough". Under QEMU/VFIO (Proxmox) the in-guest Booter step returns 0x15 (ACR_ERROR_FLCN_REG_ACCESS) instead of the response it gets on bare metal (12x 0x15, 28x 0x31 on a clean card), and retries did not help. The supported path became a host-side unlock plus cmp_no_bus_reset.ko, an empty reset_method, disable_idle_d3=1 and a GSP boot-state restore. In the documented test the guest runs a stock driver and sees 32768 MiB, 70 SM, 31/31 chunks verified. P2P was not attempted: the README row "VFIO-based passthrough Working" refers to the unlock only. The 0x15 failure is not universal to VFIO: Johnnydaboy's Proxmox fork and the lab's ESXi guest both got the patched driver to unlock inside the guest.

A longer write-up by @_j3ns (amoghmunikote on GitHub) exists, the branch's docs/PASSTHROUGH.md, since deleted from the repo; see GitHub. (Pointer: the full procedure waits for the author.)

Reading c40033b on 2026-09-07, Polskie Kaktus took it as requiring the unlock on the VM host with generic drivers in the guest, while his own working libvirt guest used the v1 unlocker inside the guest.

Reported 8 GB
Source: repo github.com/amoghmunikote/cmpunlocker by amoghmunikote on GitHub
Source: @polskiekaktus, 2026-09-07
Source: @bagstah, 2026-09-04
Source: @fafinet, 2026-09-04

PixelML club VFIO baseline

The club's earlier VFIO baseline (docs/INSTALLATION.md): Ubuntu Server 22.04 guest, Linux 6.8, Q35 + SeaBIOS (OVMF/UEFI repeatedly gave RmInitAdapter and SEC2 failures; the same cards initialized under SeaBIOS), no x-vga=1 on a compute-only card, NVIDIA 610.43.03 open module, cmpunlocker 7019bc2 with --profile=8gb. The experimental Gen2 patch was excluded because it caused guest and QEMU hangs and RCU stalls. Trap: a stock DKMS copy under updates/dkms shadowed the patched module and the card showed 8 GiB; fix with dkms remove, depmod -a, update-initramfs -u, confirm modinfo -n nvidia points at the cmpunlocker module, then a full guest shutdown. Under VFIO they had no P2P. The club later moved to bare-metal Proxmox VE 9.2 (kernel 7.0.2-6-pve) serving from a GPU LXC that shares the host driver. Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @djpandaman, 2026-10-24

SeaBIOS, not OVMF; a stock DKMS copy can shadow the patched module

The SeaBIOS rule holds across reports. Under OVMF/UEFI the club saw failures that "look like the exploit not working"; the same cards init fine under SeaBIOS. Mark Linton reported the same on Discord on 2026-09-17, two weeks before the club's docs: two 8 GB cards through VFIO in a SeaBIOS guest with rombar=0, and UEFI guest firmware breaks the unlock. Exception: Johnnydaboy's dual-CMP fork runs under OVMF (see Host or guest).

For the DKMS shadow, the exact steps:

dkms remove nvidia/610.43.03 --all
depmod -a
update-initramfs -u
Confirm modinfo -n nvidia points at updates/cmpunlocker/nvidia.ko.
Shut the guest down fully, then start it.

The club installed the driver with --kernel-module-type=open --dkms and used cmpunlocker patches 0001 to 0006. On their later bare-metal Proxmox VE 9.2 host (NVIDIA 615.71.09 open modules) the GPU LXC has static-BAR1 P2P working. The repo was posted on Discord by c133 and djpandaman.

Proven 8 GB and 10 GB
Source: repo github.com/PixelML/club-170hx by PixelML on GitHub
Source: repo github.com/Consensus-Protocol/cmp170hx by Consensus-Protocol on GitHub
Source: @darkmaniac7, 2026-08-30 (lab notes)
Source: @marklintonca, 2026-09-17

Other public VM runbooks

None of these attempt P2P; they cover the unlock and Gen2 only.

18gogogo/170hx-hosttovm-passthrough: KVM/libvirt, four cards into one VM, SeaBIOS or OVMF with pci-hole64-size=1TB. A host/guest switching runbook whose documented wedges are host-side: rmmod nvidia in D state, unbind/bind hanging while vLLM holds /dev/nvidia*, kill -9 ineffective, a reboot the only fix.
mazzanfar/cmp170hx-proxmox-pcie-gen2: Proxmox, one card, Gen2 x4 in a guest (posted 2026-09-29).
Hurricane-Computer/cmp170hx-dda-gen2: Hyper-V DDA, a Gen2 boot workaround in a Linux guest.
danvei233/170hx-pve-gen2-controller: a Proxmox Gen2 retrain controller over vsock. 丁薇 explained its design on 2026-08-20 and Studebaker pointed to it on 2026-09-01.
NeaPopa/bits-proxmox-recipes: Proxmox host driver plus LXC, BAR1 resize explicitly disabled.
The Consensus wiki lists VM passthrough as "Working (with caveats)" for the unlock only.

Toffles had Gen2 under VFIO on 2026-08-10. The dated Gen2 history of these tools is on Proxmox Gen2 passthrough.

Reported 8 GB
Source: repo github.com/18gogogo/170hx-hosttovm-passthrough by 18gogogo on GitHub
Source: repo github.com/Hurricane-Computer/cmp170hx-dda-gen2 by Hurricane-Computer on GitHub
Source: repo github.com/NeaPopa/bits-proxmox-recipes by NeaPopa on GitHub
Source: repo github.com/Consensus-Protocol/cmp170hx by Consensus-Protocol on GitHub
Source: @toffles3179, 2026-08-10
Source: repo github.com/danvei233/170hx-pve-gen2-controller by danvei233 on GitHub, posted by @dingwei0049, 2026-08-20
Source: repo github.com/danvei233/170hx-pve-gen2-controller by danvei233 on GitHub, posted by @_studebaker_, 2026-09-01
Source: repo github.com/mazzanfar/cmp170hx-proxmox-pcie-gen2 by mazzanfar on GitHub, posted by @mazzanfar7661, 2026-09-29

Gen2 stability under Proxmox passthrough

The dated history of Gen2 under Proxmox (Toffles, Dimon's hookscript, the vfio-linkd controller, SeriousRising's setup, mazzanfar's timed retrain) is on Proxmox Gen2 passthrough. Per-board results, VM rows included: Gen2 by platform.

A feature-request thread on the Discord is titled "Gen 2 Unlock not stable as PCIE Pass through Proxmox". Its only messages are Logy's: he runs the cards "in a vm : proxmox & qemu" and points anyone still having passthrough trouble to the anoane guide above. Source: @logy0, 2026-10-10
Source: @logy0, 2026-10-10 Whether Gen2 is stable for him after following it is not stated. Reported

What else is known:

The PixelML club excluded the experimental in-guest Gen2 patch after guest hangs and RCU stalls (above).
The anoane host retrains Gen2 from the host side (gen2-hammer) and reports Gen2 working.
In a register review of a Gen3 patch, ropuls noted that in an earlier sweep the value 0xA in XVE_FUSE_OVERRIDE eventually wedged the function under VFIO while 0x6 (the working Gen2 path) was stable. Source: @ropuls, 2026-10-08
Under ESXi, Gen2 works from inside the guest (below).

Why Gen2 is the hard part in a VM

The memory and compute unlock work inside a VM, but the guest cannot retrain the physical upstream PCIe port, so Gen2 needs help from the host. On 2026-07-30 there was no Gen2-in-VM support and Amogh had no machine to test VM passthrough; he said the only issue for any passthrough is the PCIe Gen2 retraining. Humvee55 said it works better with a host-side retrain and depends on the hypervisor, and Capone said the Proxmox host must retrain the link, unlike bare metal. gym reported the same split on 2026-08-02 and 2026-08-03: compute and memory unlock in the VM, no Gen2, the full unlock on bare metal. ppc64_ hit it under Proxmox PCIe passthrough on 2026-08-19. robbbbbb7973 agreed Proxmox did not work for the Gen2 unlock and relayed another member's claim that it worked under ESXi passthrough. Aeiou found TrueNAS Scale handled the GPU better than some dedicated hypervisors. Amogh flagged VM support as a to-do once he had an active cooler for a card in his own machine, and noted that kernel-level modified drivers make running the card in a primary or gaming PC awkward because of anticheat.

Reported 8 GB
Source: @._.patrik, 2026-07-30
Source: @_j3ns, 2026-07-30
Source: @scallion_9883, 2026-07-30
Source: @beefbonecapone, 2026-07-30
Source: @_____josh_____, 2026-07-30
Source: @ikyo, 2026-08-02
Source: @91d5a66bffc978331fac75b68fa86aba, 2026-08-02
Source: @aaasss1456, 2026-08-02
Source: @beefbonecapone, 2026-08-02
Source: @robbbbbb7973, 2026-08-02
Source: @jimtjames, 2026-08-03
Source: @_j3ns, 2026-08-03
Source: @ppc64_, 2026-08-19

Gen2 inside a Proxmox VM: several reports (October 2026)

Asked by Ikyo about Gen2 under Proxmox on 2026-10-03, ElectroHobo said he thought the unlocker's "v.4" fixed it and that he now runs Proxmox passthrough at PCIe Gen2. SeriousRising said dual cards with P2P working in a Proxmox VM had been running for a while, set up largely with an AI agent. Ikyo, whose Dell server lacks Gen2 and ReBAR support, asked whether it is worth using there; unanswered. Gen2 under Proxmox now has several independent reports (Toffles, Dimon, 丁薇, SeriousRising, ElectroHobo). The P2P-in-a-VM claim is a single report with no configuration or numbers posted (see P2P inside the VM).

Proven
Source: @ikyo, 2026-10-03
Source: @nathang6866, 2026-10-03
Source: @seriousrising, 2026-10-03

LXC containers instead of a VM

In a container the Proxmox host runs the patched driver and the container shares it, so nothing has to cross a passthrough boundary. The PixelML club serves this way (above).

The LXC route was proposed early

On 2026-07-30 replacedriver47 proposed LXC instead of a VM: the Proxmox host runs the NVIDIA driver with kernel modules plus the 170HX patch, and the container gets the matching userspace-only driver. Amogh confirmed LXC should work but needs work.

Reported 8 GB
Source: @replacedriver47, 2026-07-30
Source: @_j3ns, 2026-07-30

Unlocked card memtests clean inside an LXC container

SpiralGlide installed the unlock (screenshot of the expanded memory) and memtests passed with the card used inside an LXC container (2026-09-13). Initial cooling was a manual fixed duty cycle, set in the motherboard BIOS, on a loud 40 mm fan.

Reported
Source: @spiralglide, 2026-09-13

LXC passthrough: "CUDA unknown error" and connection drops mid model load (unresolved)

Josh tried LXC passthrough on Proxmox. In a TabbyAPI venv with PyTorch 2.9.0+cu128 (CUDA 12.8), torch reported "CUDA unknown error", CUDA available False, device count 5. He also saw the connection drop mid model load and wondered whether it relates to the reserved VRAM tail issue. Searching the server gave no solution, and no fix was posted.

Reported
Source: @_____josh_____, 2026-08-11
Source: @impactic, 2026-08-11

Problems reported under Proxmox

Froggo: "mine wouldnt work in proxmox". No details. Source: @froggao, 2026-10-18
ybryany could not get hardware VFIO passthrough of the CMP to work with a CachyOS (Arch) host; the card works on the host itself beside an RTX 4080 Super. Source: @ybryany, 2026-10-14
Source: @ybryany, 2026-10-14
Compute hangs after the unlock: drop bar1-resize-unlock.patch. ViPiC (Debian 13.4 guest with PCI passthrough, hypervisor not named, nvidia-open 615.71.09, also tried 610.43.02): after the unlock nvidia-smi showed 64 GB and ./verify.sh passed, but any compute workload hung the process, nvidia-smi stopped responding and the VM could not reboot cleanly; ./remove.sh --yes stopped the hangs but returned the card to 8 GB. The fix was to build the unlock without bar1-resize-unlock.patch (remove the bar1_resize entry from common/constants.yaml and the patch from PATCH_ORDER in driver/build.sh); that patch writes the card's BAR1 override registers and enables NVreg_EnableResizableBar. Without it there were no hangs and all 64 GB stayed. It had worked two days earlier, before a bootloader repair, a kernel update and a reinstall of the unlock. This matches the ESXi finding that the BAR1 resize is a Linux-host path that does not work inside a VM. Summary of his write-up; see Discord. Source: @vipic, 2026-10-18
Source: @vipic, 2026-10-18
Source: @vipic, 2026-10-18
Source: @vipic, 2026-10-19

Proxmox 9.2 (kernel 7.x) broke a working VM

Warning Dimon (2026-09-11): after updating Proxmox from 9.1.16 to 9.2.18 his VM no longer worked with the patched drivers. ST says the cause is Proxmox 9.2's 7.x kernel, which causes problems with Ampere. No fix was posted. Dimon's advice: if a Proxmox VM setup works, do not update to 9.2; the implied workaround is to stay on 9.1.x. Earlier, on 2026-08-21, HeadSpin found Proxmox "fighting with" the open driver and reinstalled Ubuntu (his unlock on Debian 13 had not failed). For contrast, the anoane recipe above runs Proxmox VE 9.2 with kernel 7.0 and a host-side unlock, and the PixelML club serves from Proxmox VE 9.2 (kernel 7.0.2-6-pve).

Source: @headspin7102, 2026-08-21
Source: @matstatman, 2026-08-21
Source: @dimon_91153, 2026-09-11
Source: @stcord, 2026-09-12

Two cards: the second fails with NvKmsKapiDevice until a PCI remove and rescan

prometasfw passed two cards to an Ubuntu 24.04.5 VM on Proxmox. After the unlock and a reboot he got [drm:nv_drm_dev_load [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000200] Failed to allocate NvKmsKapiDevice and only one card (at 64 GB) showed in nvidia-smi. His diagnosis: QEMU intercepts the PCIe bus resets the Gen2 unlock uses to force link retraining, so with two cards the second bus reset was blocked and that card did not initialize (one card retrained to Gen2, the other dropped off). Workaround: once the VM has fully booted, remove the missing card from the bus and rescan. He also had to tune the guest grub command line so the VM could initialize 128 GB of VRAM on the PCIe bus. A delay between card initializations might fix it at boot (untested); he also suggested running the retrainer on the bare-metal host and the patched driver in the VM. Humvee55 said it works fine bare metal or through vfio passthrough; Josh suggested unlocking on the host and using LXC instead.

Related reports from 2026-08-16 to 2026-09-01: on Proxmox 9.2.1 with an Ubuntu VM, passthrough, driver install and cmpunlocker all reported success, but after a reboot the VM no longer detected the GPU although lspci still listed it (the reporter offered to sponsor a bounty for working passthrough). Bagster and xPatrik could not get the Gen2 unlock working in a VM, and fiiireface had Proxmox hang after doing the unlock. Gen2 details: Proxmox Gen2 passthrough.

Reported 8 GB
Source: @prometasfw, 2026-08-10
Source: @scallion_9883, 2026-08-10
Source: @_____josh_____, 2026-08-10
Source: @mountains1605, 2026-08-10
Source: @confident_lamb_62706, 2026-08-16
Source: @bagstah, 2026-08-16
Source: @prometasfw, 2026-08-16
Source: @._.patrik, 2026-09-01
Source: @_studebaker_, 2026-09-01
Source: @giaki3003, 2026-09-01
Source: @fiiireface, 2026-09-01

Dell server + Proxmox: cards worked only at Gen1 after removing the Gen2 patch

Ikyo could only get his cards working at the base Gen1 link on a Dell server (iDRAC) running Proxmox: he had to strip the unlocker's Gen2 patch completely (on an older version of the code), and other forks did not work either. He suspected the Dell platform and ReBAR (2026-09-09). His lspci showed the SFP28 NICs linked at 8GT/s x8 with LnkCtl2 Target Link Speed 2.5GT/s, and iDRAC listed some dummy functions as Gen5 x16. Amogh asked whether other devices train above Gen1 and moved the case to a support ticket. By 2026-09-22 Ikyo had tested Proxmox with Amogh and applied the SM patch, but Gen2 and possibly ReBAR still did not work in the VM (host with 256 GB RAM). Waka324 advised plain Ubuntu Server for a single node (Proxmox gives more ways to break things), and segmond suggested Ubuntu with Linux containers.

Reported
Source: @ikyo, 2026-09-09
Source: @_j3ns, 2026-09-10
Source: @cmdrwaka324, 2026-09-22
Source: @segmond, 2026-09-22

Proxmox VM on a Dell R7625: verify says MISSING although the unlock ran

Ikyo's Proxmox VM (q35, OVMF, hostpci0 with pcie=1, serial console, 128 GB RAM, guest kernel 7.0.0-31-generic) ran the cmpunlocker verify: PCI ID 20c2, variant 8gb, expected about 65536 MiB, status MISSING (not found in nvidia-smi), result "1 GPU(s) failed unlock verification" (2026-09-09). dmesg did contain the unlock's debug log and the installed profile "8gb / geometry: 64GB", but the link retrain ended at speed 1. He considered the installer options --no-gen2-service, --no-passthrough and --no-iommu; a fresh Ubuntu install gave the same error, and he suspects the R7625 itself.

Reported 8 GB
Source: @ikyo, 2026-09-09

Passthrough-branch install failures: Debian error 2, nvidia-open not found, leftover GRID drivers

fafinet ran the passthrough branch on Debian (2026-09-08). The install failed with error 2 unless an earlier community patch was applied; with the patch, after a reboot, verify.sh failed with "Cold reboot if modules were just installed". After switching to master (Lays pointed out the merge) it still failed with error 2, and Takka guessed a hard-coded topology value that does not match the motherboard. Ikyo's passthrough install could not find nvidia-open and he saw many NVRM errors at VM startup; he suspected leftover NVIDIA GRID drivers from earlier RTX A5000 use and removed them. There was confusion over whether to install on the Proxmox host or in the VM: Amogh's instruction was the host (SeriousRising also took part).

Reported
Source: @fafinet, 2026-09-08
Source: @lays_88548, 2026-09-08
Source: @spectakkular, 2026-09-08
Source: @ikyo, 2026-09-08
Source: @seriousrising, 2026-09-08

Proxmox setups people run

ikyo: six cards, one machine with four and another with two under Proxmox. He notes the CMP is also "OK" at image generation. Source: @ikyo, 2026-10-26
Source: @ikyo, 2026-10-26
GalyGames: the unlock works on his Proxmox system (Debian based), and he reproduced Plean's 2-card Qwen3.8 Flash Next recipe on Proxmox. Source: @galygious, 2026-10-07
Source: @galygious, 2026-10-10
ElectroHobo ran 170HX cards and V100s in one Dell R730 under Proxmox while first testing, in separate VMs rather than one model across both. Source: @nathang6866, 2026-10-25 See Dell PowerEdge R730.
- He answered PJ, who had asked owners of both cards to try combining them now that 170HX prices are up. He thinks the only way to run one model across both is two VMs plus software that shards the model across machines. He also has four 32 GB SXM2 V100s (no four-card board yet) and five 170HX in a Supermicro server, and plans separate models on each group. Mixed-GPU vLLM: see Operating systems and kernels#Mixed GPUs in one host. Source: @zingy90210, 2026-10-25
  Source: @nathang6866, 2026-10-25
Mark Linton runs two 8 GB cards (64 GB each) through VFIO in a SeaBIOS guest with the Gen2 patches dropped, so the link is Gen1 x4 with no P2P (Zorg33 asked why Gen1). A longer write-up by @marklintonca exists; see Discord. (Pointer: the configuration details wait for the author.) Reported 8 GB Source: @marklintonca, 2026-09-17
Source: @zorg33, 2026-09-17

ESXi

Step-by-step for the 64 GB BAR1: BAR1 64 GB on ESXi. Code and write-up: darkmaniac7/cmp170hx-esxi. Source: repo github.com/darkmaniac7/cmp170hx-esxi by darkmaniac7 on GitHub, posted by @darkmaniac7, 2026-10-21 (lab notes)

Four 8 GB cards (10de:20c2, Hynix HBM, VBIOS 92.00.67.00.01) were passed through from VMware ESXi 8.0.3 into one Ubuntu 24.04 VM, next to other newer NVIDIA GPUs in the same guest, on a single-socket EPYC server with the slots bifurcated x8/x8 so every card sits behind its own root port. Measured in October 2026. Source: @darkmaniac7, 2026-10-21 (lab notes)

What	Status on ESXi 8 passthrough	Notes
64 GB VRAM unlock, inside the guest	works	cmpunlocker v0.5 core ported onto open modules 580.178.04. Does not need a big BAR1.
64 GiB BAR1	works, but ESXi never does it for you	needs a small UEFI app that runs before ESXi on every host boot, plus a guest-side hook on every VM power-on
PCIe Gen2	works, guest-side retrain is enough	cards come up Gen1 x8 stock, Gen2 x8 after
PCIe Gen3	fused off on these 8 GB cards	the software fuse override path is itself fused off
74 SMs	works	port of upstream PR #55 onto 580
P2P between CMPs in the VM	no	static BAR1 P2P init fails or wedges UVM in the guest; the same cards do 12/12 pairs on bare metal
Serving	4 cards run GLM-5.3-Flash W4A16 at recipe-book parity in PP4, about 0.75x in TP4	the TP4 gap is the host-staged all-reduce

Source: @darkmaniac7, 2026-10-21 (lab notes)

Earlier ESXi reports

ESXi was not first: nightdays reported the 170HX "working flawlessly" through VMware ESXi passthrough with PCIe Gen2 enabled on 2026-07-31, with no major issues. His screenshot showed a x32 link width, which poningru explained: the PCIe Gen1 to Gen5 specs allow x32 width, though physical x32 slots are very rare (replacedriver47 mentioned an Asus EPYC board). That x32 is VMware's virtual port (see Reading the link state inside the guest).

On 2026-09-07 nightdays posted a recipe for CMP 170HX cards on a Dell PowerEdge R7515 under ESXi 8. No hookscript or kernel patch was needed:

In the server BIOS, set "Memory Mapped I/O Limit" to 8TB (he says this gives enough MMIO space for the 64 GB BAR).
Pass the card through with standard passthrough.
Add the VM parameters pciPassthru.use64bitMMIO = "TRUE", pciPassthru.64bitMMIOSizeGB = "128" and hypervisor.cpuid.v0 = "FALSE".
Install Ubuntu 24.04 in the VM without third-party drivers, and snapshot the clean VM for rollback.
Run the unlock patch inside the VM, then reboot the VM.

When Ikyo asked for proof he posted screenshots and a log. Which fork and version he used was still being asked at the end of the thread. Inf could not get his cards working on ESXi (2026-08-22, no detail). The lab's survey for earlier ESXi reports covered forks, PRs and repos, not this Discord thread.

Reported 8 GB
Source: @.nightdays, 2026-07-31
Source: @poningru, 2026-07-31
Source: @replacedriver47, 2026-07-31
Source: @inf, 2026-08-22
Source: @.nightdays, 2026-09-07
Source: @ikyo, 2026-09-09

Unlock inside the guest

The six v0.5 core patches were re-applied onto pristine nvidia-open 580.178.04, because 615.71.09 plus cmpunlocker made other GPUs in the same VM hit krcWatchdog: GPU is probably locked and then wedged UVM; NVIDIA's 615.71.09 notes list that symptom as a known issue.
Each boot logs 48 Booter failed ... 0x31 lines. They are noise, not a failure.
One nvidia.ko drives every GPU in the VM. A CMP module swap or wedge takes the other GPUs down with it.
Hold the kernel and every nvidia package: an upgrade silently restores stock modules and the cards drop to 8 GB.

Source: @darkmaniac7, 2026-10-21 (lab notes)

How the guest unlock was reached

On 2026-10-17 the four cards first unlocked to 65,536 MiB on a warm module reload, on nvidia-open 615.71.09 with cmpunlocker 2d10f891 installed as --no-iommu --no-passthrough --profile=8gb. Installing 615 needed an explicit package set: the nvidia-open metapackage conflicts with the CUDA-repo 580 libraries, and libnvidia-egl-xcb1, libnvidia-egl-xlib1 and libnvidia-egl-wayland1 had to be removed first.
The 615 path was abandoned on 2026-10-18 after a clean-boot CMP CUDA hang and the watchdog wedge of the other GPUs that share the same nvidia.ko.
Accepted result: the v0.5 core backported onto open-gpu-kernel-modules 580.178.04 (Ubuntu 24.04, kernel 6.8.0-146-generic), with the SM override loop left out. All four cards report 65,536 MiB and pass a 60 GiB address-dependent write/verify on both patterns across VM boots (second window 2026-10-19: 8 x PASS). Stock 580.178.04 drops them back to 8 GiB.
No 0x15 Booter refusal was seen under ESXi, unlike the QEMU/VFIO route above.
The patched modules replace the stock DKMS copy; re-run install.sh after any kernel change, and hold the kernel image and headers with apt-mark hold. gen2.service is disabled: it does nothing in a VM.
The same 580 build later carried an HBM PLL tune (NDIV64 = 1728). nvidia-smi still shows the VBIOS nominal 1458 MHz; the real PLL value is read with the driver's hbm_mclk tool. It helped LLM serving and did nothing for compute-bound Pearl mining.

Proven 8 GB
Source: @darkmaniac7, 2026-10-21 (lab notes)
Source: repo github.com/darkmaniac7/cmp170hx-esxi by darkmaniac7 on GitHub, posted by @darkmaniac7, 2026-10-21 (lab notes)
Source: repo github.com/amoghmunikote/cmpunlocker by amoghmunikote on GitHub

Why ESXi cannot give you a 64 GB BAR1

At every host power-on the card advertises a 64 MiB BAR1: ReBAR CAP at config 0xbbc reads 0x400 and CTRL at 0xbc0 reads 0x601. Firmware sizes the BAR to that, ESXi enumerates whatever firmware assigned, and ESXi never writes the vendor override. Broadcom's VMDirectPath notes say resizable BAR is unsupported for passthrough and that devices get reset at VM power-on. Raising pciPassthru.64bitMMIOSizeGB only reserves guest address space; it cannot grow the physical BAR. The upstream bar1-resize-unlock.patch is a Linux-host path. Source: @darkmaniac7, 2026-10-21 (lab notes)

What worked: resize before ESXi boots

A small UEFI application, booted from USB (IP-KVM virtual media and a BMC boot override), runs after firmware PCI enumeration and before the ESXi loader. For each card it reserves a 64 GiB + 32 MiB span (0x1002000000) at 64 GiB alignment inside the root bridge aperture, then, with each store read back:

turns MEM and bus mastering off on the card and its parent port;
writes CYA 0x724 = 0x30;
writes CFG 0xdcc = 0x8000000a (CAP 0xbbc then goes 0x400 to 0x1ffc00);
sets ReBAR CTRL 0xbc0 to size code 16 (64 GiB), result 0x1001;
programs BAR1 and BAR3 (BAR3 is 32 MiB at base + 64 GiB);
programs the parent port's prefetchable window;
restores the parent port's COMMAND register;

then starts the existing ESXi loader with its original load options. It adds about 3.5 to 4 minutes to a host boot and has to run on every host boot. Two traps: the CYA store alone already moves CTRL from 0x801 to 0x1001, so do not expect "unchanged"; and the card's extended capability chain has a null capability (ID 0) at 0x900, so a strict decoder never reaches the ReBAR block at 0xbb0. ESXi's own view then reports BAR size 68719476736 and the guest resource1 is 64 GiB. The full procedure, build notes and risks are on BAR1 64 GB on ESXi; the code is in darkmaniac7/cmp170hx-esxi. Source: repo github.com/darkmaniac7/cmp170hx-esxi by darkmaniac7 on GitHub, posted by @darkmaniac7, 2026-10-21 (lab notes)

VM settings

pciPassthru.use64bitMMIO = TRUE, and pciPassthru.64bitMMIOSizeGB had to go from 384 to 1024 for four cards plus the other GPUs' BARs (with one 64 GiB card the guest EFI already refused the MMIO at 384). Source: @darkmaniac7, 2026-10-21 (lab notes)

How much 64-bit MMIO a VM needs

The sizes reported differ, and nobody has settled why. In the order they were posted:

nightdays (2026-09-07): Dell PowerEdge R7515 with the BIOS MMIO limit at 8TB and pciPassthru.64bitMMIOSizeGB = 128; he reported the 64 GB unlock. Source: @.nightdays, 2026-09-07
Lab (2026-10-21): with a single 64 GiB-BAR1 card the guest EFI refused 67,239,936 KiB of MMIO at 384 (the arithmetic fit, the placement did not). Four 64 GiB cards need about 544 GiB of 64-bit BAR payload once two other GPUs with 128 GiB BAR1s sit next to them, so 512 cannot work either; 1024 is the next power of two. pciPassthru.allowP2P and relaxACSforP2P are set for the other GPUs, which do P2P with each other in that VM. Source: @darkmaniac7, 2026-10-21 (lab notes)

A possible reading, not checked: if ESXi left nightdays' BAR1 at 64 MiB, as it does without a pre-boot writer, the VRAM unlock still works (it does not need the large BAR1) and 128 is enough.

Contested 8 GB

BAR1 inside the guest: the 256 MiB trap

After the host had 64 GiB BAR1s, the first VM boot showed resource1 = 64 GiB in sysfs but NVML said BAR1 Total 256 MiB. Every VM power-on resets the card's vendor override while the host allocation survives, and GSP reports what the ReBAR block says. Fix: a modprobe install nvidia hook that runs before nvidia.ko loads, maps one page of resource0 at 0x88000 (the config mirror; guest config-space writes are virtualised and never reach the device), checks the reset profile, writes 0x724 = 0x30 then 0xdcc = 0x8000000a, and requires CAP 0x1ffc00 / CTRL 0x1001 on read-back. Three lessons: a cold VM boot leaves memory decode off, so enable MEM (never bus master) first; udev runs the hook once per NVIDIA function concurrently, so take a lock; and never retry, a wrong read-back means a VM power cycle. Keep nvidia.ko out of the initramfs and never insmod by hand, or the hook is bypassed. After that NVML reports 65536 MiB BAR1 on all four cards. Source: @darkmaniac7, 2026-10-21 (lab notes)

Gen2 and Gen3 under ESXi

Stock, the cards train Gen1 x8 behind Gen4 x8 root ports. cmpunlocker's Gen2 retrain, ported as a module parameter, brings all four to Gen2 x8 inside the VM; the root-port LnkCtl2 write is a no-op in a VM (it hits VMware's virtual bridge) and turned out to be unnecessary. Gen3 is closed on these 8 GB cards: DISABLE_SW_OVERRIDE_STATUS (0x820084) reads 1 and OPT_DISABLE_GEN3_SPEED (0x820250) reads 1 on all four. One attempt through the XP/XVE registers left a card at Gen1 x8 until a VM power cycle re-ran the Gen2 retrain. Source: @darkmaniac7, 2026-10-21 (lab notes)

Reading the link state inside the guest

Under ESXi passthrough the guest's config space for the card is VMware's virtual bridge, a synthetic 5 GT/s x32 link, not the card. Take link-state verdicts from:

the card's own PCIe capability through the BAR0 config mirror (a read-only mmap of sysfs resource0; LnkCap at 0x88084, LnkCtl/LnkSta at 0x88088, LnkCtl2 at 0x880a8);
for the final word, the ESXi host's own lspci, decoding LnkCap, LnkCap2, LnkSta, LnkCtl2 and LnkSta2 of the cards and their root ports.

The cmp_pcie_speed helper in the cmp170hx-esxi repo (built from the 580.178.04 headers after lixiangwuxian/170hx_tips set_pcie_speed.c) fixes two things in that reference: it matches the card on domain, bus and slot (the guide matched slot 0 only, wrong for a card at slot 1), and it takes the RM deviceInstance from the driver's GPU ID query while the /dev minor comes from /proc/driver/nvidia/gpus/<BDF>/information, because the two differ. Never pass raw 2500/5000/8000 to SET_PCIE_SPEED.

nightdays' ESXi screenshot of 2026-07-31 showed a x32 link width, so its "Gen2 enabled" may be the synthetic 5 GT/s of VMware's virtual port rather than the card's own link: unverified.

Reported 8 GB
Source: repo github.com/darkmaniac7/cmp170hx-esxi by darkmaniac7 on GitHub, posted by @darkmaniac7, 2026-10-21 (lab notes)
Source: @.nightdays, 2026-07-31
Source: @poningru, 2026-07-31
Source: @replacedriver47, 2026-07-31

P2P inside the VM: not working

Warning 8 GB Do not expect P2P between CMPs inside an ESXi guest. With RMForceStaticBar1=1;RMPcieP2PType=1 the first plain CUDA context never exits and UVM wedges, which needed a hard VM reset (twice). Serve with NCCL_P2P_DISABLE=1 and a host-staged all-reduce, or run the cards on bare metal, where the same four cards do P2P. Source: @darkmaniac7, 2026-10-21 (lab notes)

Stock, GSP reports P2P not supported for the CMP device IDs and cudaDeviceCanAccessPeer is false on all 12 directed pairs.
RMForceStaticBar1=1 alone fails init: the static map is placed at offset 512 MiB and the framebuffer overruns the 64 GiB BAR1 (kbusEnableStaticBar1Mapping_TU102: Failed to create the static bar1 mapping).
RMForceStaticBar1=1;RMPcieP2PType=1 inits cleanly and NVML shows 64 GiB, but the first plain CUDA context never exits and UVM wedges (hard VM reset, twice).
RM only detects VMware through CPUID leaves; with hypervisor.cpuid.v0 = FALSE that detection is false. Do not gate anything on hypervisor detection in a VM that hides it.
Other GPUs in the same VM do P2P with each other (pciPassthru.allowP2P and relaxACSforP2P set), so ESXi's peer transport is not the blocker. The reading is that it is the ESXi mapping of a 64 GiB BAR1 under CUDA, not the cards; that is a reading, not a root cause. Theory
The same four cards booted into native Linux on the same host passed 12/12 directed P2P pairs.

Serving runs with NCCL_P2P_DISABLE=1 and a host-staged all-reduce. Source: @darkmaniac7, 2026-10-21 (lab notes)

More on P2P in a VM

Stock, the mailbox path moves zero bytes. On bare metal with RMForceStaticBar1=1;RMPcieP2PType=1 the same four cards pass 12/12 directed pairs with real remote loads and stores at 4 KiB, 1 MiB and 16 MiB.
In the failing guest case the first CUDA context was an 8 KiB probe with no peer step; nvidia-smi then sat in uvm_va_space_destroy.
Untested: the TRAP31 mailbox route inside a VM, a static window shrunk to the client's size, hypervisor.cpuid.v0 = TRUE.
Two cheap read-only checks before another attempt: confirm the guest's loaded nvidia_uvm srcversion matches the patched build (club-3090 hit a udev auto-load of the packaged UVM on both rigs), and confirm UVM's BAR1-peer behaviour on the driver version (from 590.44.01 UVM uses PCIe BAR1 peers automatically, faulting pre-Hopper parts with Xid 31 FAULT_UNSUPPORTED_APERTURE; 580.178.04 should not be affected).
A survey (2026-10-18/19) of all 247 readable direct cmpunlocker forks, 11 forks of forks, the upstream branches, all 65 PRs and 20+ related repos found no public report of BAR1 or mailbox P2P working, or being attempted with a patched driver, between CMP 170HX cards inside a guest on any hypervisor. Every verified CMP P2P success is on bare metal (or a Proxmox host kernel), mostly with the IOMMU off or iommu=pt and usually ACS redirect off. club-170hx: "under VFIO passthrough there is no P2P". The Consensus wiki: BAR1 peer DMA writes a raw physical address and only works if the IOMMU is not translating. NVIDIA's vGPU P2P doc notes UVM interacting with P2P on A100 in VMs, but it covers vGPU only.

Warning 8 GB
Source: repo github.com/darkmaniac7/cmp170hx-esxi by darkmaniac7 on GitHub, posted by @darkmaniac7, 2026-10-21 (lab notes)
Source: repo github.com/PixelML/club-170hx by PixelML on GitHub
Source: repo github.com/Consensus-Protocol/cmp170hx by Consensus-Protocol on GitHub

The one P2P-in-a-VM claim

The survey line holds only for verified reports. In the order posted:

DeeJay's Unraid P2P (2026-07-30) is a host install, not a guest (see Unraid below). Source: @deejay9983, 2026-07-30
Mark Linton's Proxmox VFIO guest ran with no P2P (2026-09-17). Source: @marklintonca, 2026-09-17
SeriousRising said on 2026-10-03 that dual cards with P2P working in a Proxmox VM had been running for a while, set up largely with an AI agent; no configuration or numbers were posted. Source: @seriousrising, 2026-10-03

Contested

Serving numbers under ESXi

GLM-5.3-Flash W4A16 with the DFlash2 drafter on the Morrowmake vLLM fork (engine 819146023f, recipe 1.8.0), cards at 180 W, 74-SM driver; tok/s structured / code / prose:

Layout	1 user	8 users aggregate	32k cold prefill
4x CMP, PP4 (ESXi, Gen2 x8, no P2P)	201 / 149 / 97	664 / 548 / 477	6.8 to 7.0k tok/s
4x CMP, TP4 (ESXi, Gen2 x8, no P2P)	361 / 284 / 199	697 / 587 / 529	1.6 to 1.75k tok/s
recipe book PP4 (x16, P2P on)	232 / 210 / 100	664 / 609 / 469	7.4k tok/s
recipe book TP4 (x16, P2P on)	483 / 442 / 232	944 / 835 / 667	3.2k tok/s

PP4 lands on the recipe book within noise on decode; TP4 is about 0.75x on decode and 0.55x on prefill, which is the all-reduce going through host memory over Gen2 x8 instead of P2P over x16. Source: @darkmaniac7, 2026-10-21 (lab notes)

ESXi traps

nvidia-persistenced is down after a VM power cycle; start it before any docker run that uses the NVIDIA runtime.
Power caps reset to 250 W on every module load. Set them after.
Only one NVreg_RegistryDwordsPerDevice line counts; a second file silently replaces the first.
During a UVM wedge do not run nvidia-smi: every new caller blocks in D state. Hard reset the VM after restoring known-good modules on disk.
Never follow a successful 64 GiB host boot with a "normal" reboot just to check something; you lose the allocation and have to run the pre-boot step again.

Source: @darkmaniac7, 2026-10-21 (lab notes)

Other hypervisor notes

vGPU is in progress in cmpunlocker: Amogh announced on 2026-10-26 that vGPU is close to working, on the vGPU-and-others branch, not in main. Theory Source: @_j3ns, 2026-10-26 See Unlocks.
Mixed host. A CMP in a desktop that also drives a gaming GPU works on bare metal (ybryany, CachyOS, RTX 4080 Super), but the global Gen2 registry keys would also force the other card to Gen2, so he left them off. Source: @ybryany, 2026-10-14 See Operating systems and kernels#Mixed GPUs in one host.
- Mixing CMP variants in one host (2026-08-14): quark2world thinks the driver detects each card's PCI ID and applies the matching patch, so 64 GB and 40 GB cards could coexist, but he had not heard of anyone doing it. On 2026-08-22 princess_melie noted a V100 needs the 580 branch and nothing existed then for the 170HX unlock on 580 (the ESXi guest above later ran the unlock on 580.178.04). Reported Source: @schakaqt, 2026-08-14
  Source: @quark2world, 2026-08-14
  Source: @princess_melie, 2026-08-22
Agents in VMs. Not a passthrough issue, but people running agent workloads in VMs on these hosts report models probing networks and acting destructively; sandbox them. Source: @segmond, 2026-10-20

Unraid: unlock, Gen2 and P2P on the host with Unraid-specific hacks

DeeJay verified the unlock, Gen2 and P2P end to end on Unraid with kernel 6.18.33 (2026-07-30), using cmpunlocker v0.2 plus aikitoria's P2P patch. This is an Unraid host install, not a guest. It needed Unraid-specific hacks: a RAM-backed OS, no kernel build tree, and a Module.symvers built from kallsyms; a mainstream distro with real kernel headers makes most of them unnecessary. He was having Claude write an install script that runs both (MrManny asked for an option to choose the number of cards) and merged the fixes into his own repo for his rented rigs so setup is a single download. Drack99 later asked whether anyone runs the 170HX on an Unraid host, assuming a kernel rebuild with the patches (2026-08-27), and BVls asked whether the unlock is possible and stable in a VM on Unraid; neither got an answer.

Reported 8 GB
Source: @deejay9983, 2026-07-30
Source: @mrmannyatx, 2026-07-30
Source: @drack99, 2026-08-27
Source: @bvls7857, 2026-09-25

XCP-ng: the passed-through card stays at Gen1

shad passed a card through from an XCP-ng host and ran the unlock in the guest; the link showed PCIe Gen1, and he concluded the card cannot be used in XCP-ng with PCIe passthrough to VMs (2026-08-23). alexdns recommended QEMU/KVM instead, and shad was considering Proxmox. On 2026-09-22 chonsworth said he was on a new server with a disposable VM and needed to work out how to get the unlock to take under XCP-ng, which implies it still did not work out of the box.

Reported
Source: @sh.ad, 2026-08-23
Source: @1234444444_, 2026-08-24
Source: @chonsworth, 2026-09-22

libvirt/virt-manager: only the v1 unlocker worked inside the guest

Polskie Kaktus got the unlock working in a KVM guest under virt-manager/libvirt only with the v1 unlocker (posted earlier by Bagster) installed inside the guest VM (2026-09-07). The host binds the 170HX to vfio-pci at boot with a modprobe config. Reading master commit c40033b, newer versions appear to require the unlock on the VM host with generic drivers in the guest; he had not tested that. He did not remember needing pci=realloc (Ikyo was checking whether it is needed) but might try it to get Resizable BAR working.

Reported
Source: @polskiekaktus, 2026-09-07
Source: @ikyo, 2026-09-08

CMP 90HX in a Windows VM

This entry is about the CMP 90HX, not the 170HX; see Other CMP cards and other NVIDIA GPUs. ICY runs 90HX cards at Gen2 x16 under Arch Linux with Windows in QEMU through GPU passthrough. Performance is about RTX 3060 level: Stalker 2, Atomic Heart and Helldivers at medium settings in FHD with a 200 W power limit gave 70 to 90 FPS. emogop's card at PCIe 1.1 x4 performed about like a GTX 1060. emogop noted the 90HX is heavily power-limit bound: its GDDR6X draws about 80 W at max clocks, leaving about 120 W for the GA102 at a 200 W cap, so undervolting helps. ICY kept 200 W because four cards on a 1400 W PSU would exceed it otherwise.

Reported
Source: @icych, 2026-08-31
Source: @emogop, 2026-08-31

Open questions

Gen2 under Proxmox passthrough now has several working reports with a host-side retrain (above, and Proxmox Gen2 passthrough); a long-run stability report with the current cmpunlocker, across host reboots and power cycles, is still missing.
P2P between CMPs inside any hypervisor: no verified report of it working (one unconfirmed Proxmox claim, 2026-10-03).
TRAP31 mailbox P2P inside a VM, a smaller static BAR1 window, and hypervisor.cpuid.v0 = TRUE under ESXi are untested. Source: @darkmaniac7, 2026-10-21 (lab notes)

Low PCI power states, nested passthrough, macOS eGPU passthrough

barackorama, planning to pass his card through to one or two VMs, asked how the 170HX behaves coming out of lower PCI power states (2026-07-30); nobody answered, and Studebaker said VMs were even worse than bare metal for his crash-heavy experiments. fafinet asked whether MIG or vGPU would allow nested passthrough, a VM inside a VM using the CMP (2026-09-15); unanswered. ST noted Macs now allow eGPU passthrough into a VM and planned to try a CMP through a PCIe switch that way (2026-09-30); untested.

Theory
Source: @_studebaker_, 2026-07-29
Source: @barackorama, 2026-07-30
Source: @fafinet, 2026-09-15
Source: @stcord, 2026-09-30

One card split across several VMs

Ikyo asked whether one card could be split across several VMs, vGPU style (2026-08-02); nobody answered. vGPU work is in progress upstream (see Other hypervisor notes).

Theory
Source: @ikyo, 2026-08-02