Virtual machines and passthrough

Reported 8 GB and 10 GB

Running CMP 170HX cards inside a virtual machine works, with more moving parts than bare metal. The short version:

Unlock where the NVIDIA driver runs. Under Proxmox/QEMU most people unlock on the host and pass the unlocked card in. Under ESXi the unlock runs inside the guest.
64 GB VRAM works in a VM. It does not need a big BAR1.
Gen2 works in a VM, but how it is done differs per hypervisor, and the Gen2 path has caused guest hangs under Proxmox for some people.
64 GB BAR1 in a VM is hard (ESXi needs a pre-boot step on every host boot).
P2P between CMPs inside a VM has not been shown to work by anyone in these sources.
Bare metal is much simpler. Source: @darkmaniac7, 2026-10-29 (lab notes)

Where to run the unlock

Asked whether the unlock must happen on the hypervisor, benoit tried both on Proxmox: unlocking fully inside the VM gave him trouble unlocking both cards, while unlocking on the Proxmox host worked and is what he kept. Source: @jack0x01, 2026-10-08
Source: @bismuth67, 2026-10-08 SpiralGlide does the same: unlock on the host and pass the card into a container that runs Docker containers. Proven Source: @spiralglide, 2026-10-08

The PixelML club writes the same rule from its VFIO baseline: under VFIO the unlock can be applied host-side so the guest runs the stock driver; their second host showed Gen2 x16 and 64 GB in the guest that way. Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @djpandaman, 2026-10-24 (docs/INSTALLATION.md)

The cmpunlocker upstream "Virtual Passthrough" route also unlocks on the host, because under QEMU/VFIO the in-guest Booter path returns 0x15 (ACR_ERROR_FLCN_REG_ACCESS) and the guest ends up on a stock driver. Under ESXi that error did not appear and the whole v0.5 core ran inside the guest. Source: @darkmaniac7, 2026-10-21 (lab notes)

aidoc confirmed his register probe script can be run from inside a VM. Source: @sekkit_75019, 2026-10-04

Proxmox (QEMU/KVM, VFIO)

A working Proxmox host recipe

The most complete public Proxmox write-up is the anoane repo (MIT license), linked by Logy as the guide he followed: Source: repo github.com/anoane/01-proxmox-dual-gpu-passthrough by anoane on GitHub, posted by @logy0, 2026-10-10. It passes an RTX PRO 6000 Blackwell and an 8 GB CMP 170HX unlocked to 64 GB into one Ubuntu 24.04 guest. The author calls it a point-in-time transcription of one host, not a turnkey script. Key points from the repo (commit 865a608, 2026-09-22):

Host: Proxmox VE 9.2.x, kernel 7.0.x-pve, Secure Boot disabled (mokutil --sb-state must say disabled), AMD Ryzen 9 9950X3D. cmpunlocker pinned at commit 76f0954 on the host with driver 610.43.02. The guest runs Ubuntu 24.04.4, kernel 6.8.0-139-generic, the stock NVIDIA driver 610.43.02; the unlock lives on the host. The guest sees the CMP as "NVIDIA Graphics Device" with 65,536 MiB, so match it by PCI id 10de:20c2, not by name.
Host kernel command line: quiet amd_iommu=on iommu=pt; modules vfio vfio_iommu_type1 vfio_pci vfio_virqfd.
options vfio-pci disable_idle_d3=1: if vfio idles the CMP into D3, BAR0 reads all ones and the card disappears.
The CMP is armed and bound by cmpunlocker's boot service, not by the vfio-pci ids= list.
VM: machine q35 (required), bios seabios, cpu host, balloon 0, CMP passed as hostpci1: 0000:06:00.0,pcie=1,rombar=0. The CMP shared an IOMMU group with a PCIe bridge, so the whole group's function set is passed.
A pre-start hookscript re-runs cmpunlocker's gsp-restore for cards on vfio-pci before every VM start, because the vfio bind event only fires on a real driver change and later VM starts would otherwise skip the restore.
A VM that is killed rather than shut down leaves the GSP boot state latched and the next VM gets no GPU; only ./tools/passthrough.sh restore <bdf> clears it. cmp_no_bus_reset.ko blocks the bus reset vfio does at bind, which would otherwise erase the unlock.
PCIe: the card trains Gen1 at boot. A host-side gen2-hammer script walks the upstream bridge and retrains with setpci until LnkSta reads Gen2, run by a gen2.service before basic.target. The card was confirmed Gen2 x16 in a PCIe 5.0 x16 slot; in the documented build it sits in a PCIe 3.0 x1 slot and measured about 0.38 GB/s.

From the repo README and vm/ai-vm.conf: Source: repo github.com/anoane/01-proxmox-dual-gpu-passthrough by anoane on GitHub, posted by @logy0, 2026-10-10

PixelML club VFIO baseline

The club's earlier VFIO baseline (docs/INSTALLATION.md): Ubuntu Server 22.04 guest, Linux 6.8, Q35 + SeaBIOS (OVMF/UEFI repeatedly gave RmInitAdapter and SEC2 failures; the same cards initialized under SeaBIOS), no x-vga=1 on a compute-only card, NVIDIA 610.43.03 open module, cmpunlocker 7019bc2 with --profile=8gb. The experimental Gen2 patch was excluded because it caused guest and QEMU hangs and RCU stalls. Trap: a stock DKMS copy under updates/dkms shadowed the patched module and the card showed 8 GiB; fix with dkms remove, depmod -a, update-initramfs -u, confirm modinfo -n nvidia points at the cmpunlocker module, then a full guest shutdown. Under VFIO they had no P2P. The club later moved to bare-metal Proxmox VE 9.2 (kernel 7.0.2-6-pve) serving from a GPU LXC that shares the host driver. Source: repo github.com/PixelML/club-170hx by PixelML on GitHub, posted by @djpandaman, 2026-10-24

Gen2 stability under Proxmox passthrough

A feature-request thread on the Discord is titled "Gen 2 Unlock not stable as PCIE Pass through Proxmox". Its only messages are Logy's: he runs the cards "in a vm : proxmox & qemu" and points anyone still having passthrough trouble to the anoane guide above. Source: @logy0, 2026-10-10
Source: @logy0, 2026-10-10 Whether Gen2 is stable for him after following it is not stated. Reported

What else is known:

The PixelML club excluded the experimental in-guest Gen2 patch after guest hangs and RCU stalls (above).
The anoane host retrains Gen2 from the host side (gen2-hammer) and reports Gen2 working.
In a register review of a Gen3 patch, ropuls noted that in an earlier sweep the value 0xA in XVE_FUSE_OVERRIDE eventually wedged the function under VFIO while 0x6 (the working Gen2 path) was stable. Source: @ropuls, 2026-10-08
Under ESXi, Gen2 works from inside the guest (below).

Problems reported under Proxmox

Froggo: "mine wouldnt work in proxmox". No details. Source: @froggao, 2026-10-18
ybryany could not get hardware VFIO passthrough of the CMP to work with a CachyOS (Arch) host; the card works on the host itself beside an RTX 4080 Super. Source: @ybryany, 2026-10-14
Source: @ybryany, 2026-10-14
Compute hangs in a passthrough VM after the unlock: A longer write-up by @vipic exists; see Discord.

Proxmox setups people run

ikyo: six cards, one machine with four and another with two under Proxmox. Source: @ikyo, 2026-10-26
GalyGames: the unlock works on his Proxmox system (Debian based), and he reproduced Plean's 2-card Qwen3.8 Flash Next recipe on Proxmox. Source: @galygious, 2026-10-07
Source: @galygious, 2026-10-10
ElectroHobo ran 170HX cards and V100s in one Dell R730 under Proxmox while first testing, in separate VMs rather than one model across both. Source: @nathang6866, 2026-10-25 See Dell PowerEdge R730.

ESXi

Four 8 GB cards (10de:20c2, Hynix HBM, VBIOS 92.00.67.00.01) were passed through from VMware ESXi 8.0.3 into one Ubuntu 24.04 VM, next to other newer NVIDIA GPUs in the same guest, on a single-socket EPYC server with the slots bifurcated x8/x8 so every card sits behind its own root port. Measured in October 2026. Source: @darkmaniac7, 2026-10-21 (lab notes)

What	Status on ESXi 8 passthrough	Notes
64 GB VRAM unlock, inside the guest	works	cmpunlocker v0.5 core ported onto open modules 580.178.04. Does not need a big BAR1.
64 GiB BAR1	works, but ESXi never does it for you	needs a small UEFI app that runs before ESXi on every host boot, plus a guest-side hook on every VM power-on
PCIe Gen2	works, guest-side retrain is enough	cards come up Gen1 x8 stock, Gen2 x8 after
PCIe Gen3	fused off on these 8 GB cards	the software fuse override path is itself fused off
74 SMs	works	port of upstream PR #55 onto 580
P2P between CMPs in the VM	no	static BAR1 P2P init fails or wedges UVM in the guest; the same cards do 12/12 pairs on bare metal
Serving	4 cards run GLM-5.3-Flash W4A16 at recipe-book parity in PP4, about 0.75x in TP4	the TP4 gap is the host-staged all-reduce

Source: @darkmaniac7, 2026-10-21 (lab notes)

Unlock inside the guest

The six v0.5 core patches were re-applied onto pristine nvidia-open 580.178.04, because 615.71.09 plus cmpunlocker made other GPUs in the same VM hit krcWatchdog: GPU is probably locked and then wedged UVM; NVIDIA's 615.71.09 notes list that symptom as a known issue.
Each boot logs 48 Booter failed ... 0x31 lines. They are noise, not a failure.
One nvidia.ko drives every GPU in the VM. A CMP module swap or wedge takes the other GPUs down with it.
Hold the kernel and every nvidia package: an upgrade silently restores stock modules and the cards drop to 8 GB.

Source: @darkmaniac7, 2026-10-21 (lab notes)

Why ESXi cannot give you a 64 GB BAR1

At every host power-on the card advertises a 64 MiB BAR1: ReBAR CAP at config 0xbbc reads 0x400 and CTRL at 0xbc0 reads 0x601. Firmware sizes the BAR to that, ESXi enumerates whatever firmware assigned, and ESXi never writes the vendor override. Broadcom's VMDirectPath notes say resizable BAR is unsupported for passthrough and that devices get reset at VM power-on. Raising pciPassthru.64bitMMIOSizeGB only reserves guest address space; it cannot grow the physical BAR. The upstream bar1-resize-unlock.patch is a Linux-host path. Source: @darkmaniac7, 2026-10-21 (lab notes)

What worked: resize before ESXi boots

A small UEFI application, booted from USB (IP-KVM virtual media and a BMC boot override), runs after firmware PCI enumeration and before the ESXi loader. For each card it reserves a 64 GiB + 32 MiB span (0x1002000000) at 64 GiB alignment inside the root bridge aperture, then, with each store read back:

turns MEM and bus mastering off on the card and its parent port;
writes CYA 0x724 = 0x30;
writes CFG 0xdcc = 0x8000000a (CAP 0xbbc then goes 0x400 to 0x1ffc00);
sets ReBAR CTRL 0xbc0 to size code 16 (64 GiB), result 0x1001;
programs BAR1 and BAR3 (BAR3 is 32 MiB at base + 64 GiB);
programs the parent port's prefetchable window;
restores the parent port's COMMAND register;

then starts the existing ESXi loader with its original load options. It adds about 3.5 to 4 minutes to a host boot and has to run on every host boot. Two traps: the CYA store alone already moves CTRL from 0x801 to 0x1001, so do not expect "unchanged"; and the card's extended capability chain has a null capability (ID 0) at 0x900, so a strict decoder never reaches the ReBAR block at 0xbb0. ESXi's own view then reports BAR size 68719476736 and the guest resource1 is 64 GiB. Ask @darkmaniac7 on Discord for the writer before improvising one. Source: @darkmaniac7, 2026-10-21 (lab notes)

VM settings

pciPassthru.use64bitMMIO = TRUE, and pciPassthru.64bitMMIOSizeGB had to go from 384 to 1024 for four cards plus the other GPUs' BARs (with one 64 GiB card the guest EFI already refused the MMIO at 384). Source: @darkmaniac7, 2026-10-21 (lab notes)

BAR1 inside the guest: the 256 MiB trap

After the host had 64 GiB BAR1s, the first VM boot showed resource1 = 64 GiB in sysfs but NVML said BAR1 Total 256 MiB. Every VM power-on resets the card's vendor override while the host allocation survives, and GSP reports what the ReBAR block says. Fix: a modprobe install nvidia hook that runs before nvidia.ko loads, maps one page of resource0 at 0x88000 (the config mirror; guest config-space writes are virtualised and never reach the device), checks the reset profile, writes 0x724 = 0x30 then 0xdcc = 0x8000000a, and requires CAP 0x1ffc00 / CTRL 0x1001 on read-back. Three lessons: a cold VM boot leaves memory decode off, so enable MEM (never bus master) first; udev runs the hook once per NVIDIA function concurrently, so take a lock; and never retry, a wrong read-back means a VM power cycle. Keep nvidia.ko out of the initramfs and never insmod by hand, or the hook is bypassed. After that NVML reports 65536 MiB BAR1 on all four cards. Source: @darkmaniac7, 2026-10-21 (lab notes)

Gen2 and Gen3 under ESXi

Stock, the cards train Gen1 x8 behind Gen4 x8 root ports. cmpunlocker's Gen2 retrain, ported as a module parameter, brings all four to Gen2 x8 inside the VM; the root-port LnkCtl2 write is a no-op in a VM (it hits VMware's virtual bridge) and turned out to be unnecessary. Gen3 is closed on these 8 GB cards: DISABLE_SW_OVERRIDE_STATUS (0x820084) reads 1 and OPT_DISABLE_GEN3_SPEED (0x820250) reads 1 on all four. One attempt through the XP/XVE registers left a card at Gen1 x8 until a VM power cycle re-ran the Gen2 retrain. Source: @darkmaniac7, 2026-10-21 (lab notes)

P2P inside the VM: not working

Stock, GSP reports P2P not supported for the CMP device IDs and cudaDeviceCanAccessPeer is false on all 12 directed pairs.
RMForceStaticBar1=1 alone fails init: the static map is placed at offset 512 MiB and the framebuffer overruns the 64 GiB BAR1 (kbusEnableStaticBar1Mapping_TU102: Failed to create the static bar1 mapping).
RMForceStaticBar1=1;RMPcieP2PType=1 inits cleanly and NVML shows 64 GiB, but the first plain CUDA context never exits and UVM wedges (hard VM reset, twice).
RM only detects VMware through CPUID leaves; with hypervisor.cpuid.v0 = FALSE that detection is false. Do not gate anything on hypervisor detection in a VM that hides it.
Other GPUs in the same VM do P2P with each other (pciPassthru.allowP2P and relaxACSforP2P set), so ESXi's peer transport is not the blocker. The reading is that it is the ESXi mapping of a 64 GiB BAR1 under CUDA, not the cards; that is a reading, not a root cause. Theory
The same four cards booted into native Linux on the same host passed 12/12 directed P2P pairs.

Serving runs with NCCL_P2P_DISABLE=1 and a host-staged all-reduce. Source: @darkmaniac7, 2026-10-21 (lab notes)

Serving numbers under ESXi

GLM-5.3-Flash W4A16 with the DFlash2 drafter on the Morrowmake vLLM fork (engine 819146023f, recipe 1.8.0), cards at 180 W, 74-SM driver; tok/s structured / code / prose:

Layout	1 user	8 users aggregate	32k cold prefill
4x CMP, PP4 (ESXi, Gen2 x8, no P2P)	201 / 149 / 97	664 / 548 / 477	6.8 to 7.0k tok/s
4x CMP, TP4 (ESXi, Gen2 x8, no P2P)	361 / 284 / 199	697 / 587 / 529	1.6 to 1.75k tok/s
recipe book PP4 (x16, P2P on)	232 / 210 / 100	664 / 609 / 469	7.4k tok/s
recipe book TP4 (x16, P2P on)	483 / 442 / 232	944 / 835 / 667	3.2k tok/s

PP4 lands on the recipe book within noise on decode; TP4 is about 0.75x on decode and 0.55x on prefill, which is the all-reduce going through host memory over Gen2 x8 instead of P2P over x16. Source: @darkmaniac7, 2026-10-21 (lab notes)

ESXi traps

nvidia-persistenced is down after a VM power cycle; start it before any docker run that uses the NVIDIA runtime.
Power caps reset to 250 W on every module load. Set them after.
Only one NVreg_RegistryDwordsPerDevice line counts; a second file silently replaces the first.
During a UVM wedge do not run nvidia-smi: every new caller blocks in D state. Hard reset the VM after restoring known-good modules on disk.
Never follow a successful 64 GiB host boot with a "normal" reboot just to check something; you lose the allocation and have to run the pre-boot step again.

Source: @darkmaniac7, 2026-10-21 (lab notes)

Other hypervisor notes

vGPU is in progress in cmpunlocker: Amogh announced on 2026-10-26 that vGPU is close to working, on the vGPU-and-others branch, not in main. Theory Source: @_j3ns, 2026-10-26 See Unlocks.
Mixed host. A CMP in a desktop that also drives a gaming GPU works on bare metal (ybryany, CachyOS, RTX 4080 Super), but the global Gen2 registry keys would also force the other card to Gen2, so he left them off. Source: @ybryany, 2026-10-14 See Operating systems and kernels#Mixed GPUs in one host.
Agents in VMs. Not a passthrough issue, but people running agent workloads in VMs on these hosts report models probing networks and acting destructively; sandbox them. Source: @segmond, 2026-10-20

Open questions

Is Gen2 stable under Proxmox passthrough with the current cmpunlocker, using a host-side retrain? Only the thread title and the anoane recipe speak to it.
P2P between CMPs inside any hypervisor: no public report of it working.
TRAP31 mailbox P2P inside a VM, a smaller static BAR1 window, and hypervisor.cpuid.v0 = TRUE under ESXi are untested. Source: @darkmaniac7, 2026-10-21 (lab notes)