Resizable BAR and BAR1 64 GB: Difference between revisions

Revision as of 08:49, 30 October 2026

Resizable BAR / BAR1 64 GB
Reported 8 GB and 10 GB
Driver	cmpunlocker-patched driver
Unlock build	amoghmunikote/cmpunlocker (bar1-resize-unlock.patch, credit asm64-hooligan / JP)
Steps	Resizable BAR and BAR1 64 GB
Source	link
Confirmations	None yet.

The stock card exposes BAR1 at 64 MiB. cmpunlocker can resize it to 64 GB, which is what large-BAR GPU-to-GPU P2P uses. It is in the README feature table as Working, credited to asm64-hooligan (JP). Whether it actually allocates depends on the motherboard.

How it works

bar1-resize-unlock.patch runs in nv_pci_probe for PCI IDs 0x20C2 and 0x2082: it maps the XVE registers at BAR0 + 0x88000, writes CYA +0x724 = 0x30 and CFG +0xDCC = 0x8000000A, then sets NVreg_EnableResizableBar = 1 so the driver attempts up to 64 GB. (From the cmpunlocker patch source.) asm64's fork does the same as patch 0009.

The board must provide the space. You need Above 4G decoding and a large 64-bit MMIO window above 4 GB; a 64 GB BAR request fails on a board that only opens a 2-16 GB window.

Boards where BAR1 will not allocate

ASUS WRX90 and Trust Inter's board: "BAR1, I can't arrange, no space"; no fix found, the agent ended up

 suggesting an ASUS BIOS-mod ticket. Source: @trustinter, 2026-10-07

Minisforum MS-01 (cards behind a PLX 88096): BAR1 stuck at 64 MB even with Resizable BAR enabled; MMIO is 256 GB

 but the BIOS tries to place the BARs in a 512 MB section. The PLX 88096 firmware needs hotplug enabled to remap BAR
 addressing. Source: @replacedriver47, 2026-10-24
 Same symptom reported on ASUS WRX80E and WRX90. Source: @replacedriver47, 2026-10-10

ASUS P9X79 WS (X79): the BIOS allocates only a small window above 4 GB (its agent put it at 2-16 GB), so the

 64 GB BAR fails. fdelfini's board already had Above 4G decoding; his agent changed Ubuntu kernel parameters to enlarge
 the MMIOH window and it then worked. The exact parameters were not posted, so this is an OS-side MMIOH-window fix to
 reproduce, not a recipe yet. Source: @fdelfini, 2026-10-19

When the board cannot open the 64 GB BAR, use the mailbox P2P path instead, which runs inside the stock 64 MB BAR. Source: @ob88594, 2026-10-05

Known issue: VM compute hang

Warning In a Debian passthrough VM, after the unlock reported 64 GB and verify passed, any compute workload hung the process and nvidia-smi stopped responding. The fix was to drop bar1-resize-unlock.patch (remove the bar1_resize entry from common/constants.yaml and the patch from the PATCH_ORDER array in driver/build.sh), rebuild, and all 64 GB remained with no hangs. That patch writes the XVE BAR1 registers for PCI IDs 0x20C2 and 0x2082 and enables NVreg_EnableResizableBar; without it there are no hangs and the card keeps 64 GB. The hang appeared only after a bootloader repair, kernel update and cmpunlocker reinstall; ViPiC suspected the hypervisor not fully resetting the card between VM reboots. Source: @vipic, 2026-10-18

Verify

nvidia-smi -q | grep -A3 BAR1
lspci -s <bdf> -vvv | grep -i "Region 1"

Status

Works on boards that expose a large enough MMIO window; fails on several HEDT/PLX boards with no universal fix, so Reported, not Proven. Needs Above 4G decoding in BIOS.

@@ Line 32: / Line 32: @@
    addressing. {{Source|replacedriver47|2026-10-24|https://discord.com/channels/1518634379604136178/1534370835220004885/1563580726882861097}}
    Same symptom reported on '''ASUS WRX80E''' and WRX90. {{Source|replacedriver47|2026-10-10|https://discord.com/channels/1518634379604136178/1518695289278566450/1558565136980254791}}
-* '''ASUS P9X79 WS''' (X79): the BIOS allocates only a small window above 4 GB, so the 64 GB BAR fails; fixed from the OS
+* '''ASUS P9X79 WS''' (X79): the BIOS allocates only a small window above 4 GB (its agent put it at 2-16 GB), so the
-  side by changing kernel parameters to enlarge the MMIOH window (exact params not captured; a longer write-up by
+GB BAR fails. fdelfini's board already had Above 4G decoding; his agent changed Ubuntu kernel parameters to enlarge
-   @fdelfini exists, see Discord). {{Source|fdelfini|2026-10-19|https://discord.com/channels/1518634379604136178/1518692084729511967/1561810276351741975}}
+  the MMIOH window and it then worked. The exact parameters were not posted, so this is an OS-side MMIOH-window fix to
+   reproduce, not a recipe yet. {{Source|fdelfini|2026-10-19|https://discord.com/channels/1518634379604136178/1518692084729511967/1561810276351741975}}
 When the board cannot open the 64 GB BAR, use the mailbox [[P2P]] path instead, which runs inside the stock 64 MB BAR.
@@ Line 45: / Line 46: @@
 <code>bar1-resize-unlock.patch</code>''' (remove the <code>bar1_resize</code> entry from
 <code>common/constants.yaml</code> and the patch from the <code>PATCH_ORDER</code> array in
-<code>driver/build.sh</code>), rebuild, and all 64 GB remained with no hangs. A longer write-up by @vipic exists; see
+<code>driver/build.sh</code>), rebuild, and all 64 GB remained with no hangs. That patch writes the XVE BAR1 registers for
-Discord. {{Source|vipic|2026-10-18|https://discord.com/channels/1518634379604136178/1518692084729511967/1561429038361223329}}
+PCI IDs <code>0x20C2</code> and <code>0x2082</code> and enables <code>NVreg_EnableResizableBar</code>; without it there
+are no hangs and the card keeps 64 GB. The hang appeared only after a bootloader repair, kernel update and cmpunlocker
+reinstall; ViPiC suspected the hypervisor not fully resetting the card between VM reboots.
+{{Source|vipic|2026-10-18|https://discord.com/channels/1518634379604136178/1518692084729511967/1561429038361223329}}
 == Verify ==