Platforms - Revision history

ImportBot: Overview of boards, servers and hosts reported with CMP 170HX cards (Gen2, BAR1, P2P, cards per host), from Discord platform reports, PixelML/club-170hx, anoane/01-proxmox-dual-gpu-passthrough and @darkmaniac7 (Ivan) lab notes

2026-10-30T10:02:24Z

Overview of boards, servers and hosts reported with CMP 170HX cards (Gen2, BAR1, P2P, cards per host), from Discord platform reports, PixelML/club-170hx, anoane/01-proxmox-dual-gpu-passthrough and @darkmaniac7 (Ivan) lab notes

← Older revision		Revision as of 10:02, 30 October 2026
Line 87:		Line 87:
	** Miséricorde: RDIMMs do not need to be matched; you can even mix ICs (he bought 11 used 16 GB sticks at 40 USD each for an EPYC Milan build). {{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813031363973225}}<br />{{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813484072247297}}<br />{{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813506113306756}}		** Miséricorde: RDIMMs do not need to be matched; you can even mix ICs (he bought 11 used 16 GB sticks at 40 USD each for an EPYC Milan build). {{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813031363973225}}<br />{{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813484072247297}}<br />{{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813506113306756}}
	** endsjeremy: he insists on type and revision matching, especially with used dual-rank memory. {{Source\|handle=endsjeremy\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813660652314675}}		** endsjeremy: he insists on type and revision matching, especially with used dual-rank memory. {{Source\|handle=endsjeremy\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813660652314675}}
	* '''CPU cooler:''' GlobalApathy ~~and Dc both use~~ the Arctic SP3 cooler ~~on their EPYC hosts~~; Dc: it keeps the CPU cool and is silent. {{Source\|handle=globalapathy\|date=2026-10-11\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1558682489017335870}}<br />{{Source\|handle=daveedcoles\|date=2026-10-11\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1558694978979500043}}		* '''CPU cooler:''' GlobalApathy rates the Arctic cooler for SP3 (EPYC) as pretty good. He recognised the same Arctic cooler in Dc's build photo; Dc did not name the model, and says it keeps the CPU cool and is silent. {{Source\|handle=globalapathy\|date=2026-10-11\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1558682489017335870}}<br />{{Source\|handle=daveedcoles\|date=2026-10-11\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1558694978979500043}}<br />{{Source\|handle=globalapathy\|date=2026-10-14\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1559752062852669461}}
	* '''Renting the box out:''' endsjeremy suggests about 2:1 system RAM to VRAM and more than 16 cores for rentals. {{Source\|handle=endsjeremy\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557504597457903700}}<br />{{Source\|handle=endsjeremy\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557504862894424179}}		* '''Renting the box out:''' endsjeremy suggests about 2:1 system RAM to VRAM and more than 16 cores for rentals. {{Source\|handle=endsjeremy\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557504597457903700}}<br />{{Source\|handle=endsjeremy\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557504862894424179}}
	* '''Broadwell CPUs''' may halve some Qwen Flash recipe numbers (ElectroHobo); segmond attributes the same gap to no x16 and no P2P. {{Status\|Theory}} {{Source\|handle=segmond\|date=2026-10-20\|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1561950054254645291}}<br />{{Source\|handle=nathang6866\|date=2026-10-20\|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1561951080882114701}}		* '''Broadwell CPUs''' may halve some Qwen Flash recipe numbers (ElectroHobo); segmond attributes the same gap to no x16 and no P2P. {{Status\|Theory}} {{Source\|handle=segmond\|date=2026-10-20\|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1561950054254645291}}<br />{{Source\|handle=nathang6866\|date=2026-10-20\|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1561951080882114701}}

ImportBot: Overview of boards, servers and hosts reported with CMP 170HX cards (Gen2, BAR1, P2P, cards per host), from Discord platform reports, PixelML/club-170hx, anoane/01-proxmox-dual-gpu-passthrough and @darkmaniac7 (Ivan) lab notes

2026-10-30T08:47:26Z

← Older revision		Revision as of 08:47, 30 October 2026
Line 108:		Line 108:
	== Suggested, no CMP result yet ==		== Suggested, no CMP result yet ==
	These come up as hosts but nobody has posted a CMP result on them. Good candidates for testing tasks on the board.		These come up as hosts but nobody has posted a CMP result on them. Good candidates for testing tasks on the board.
	* Gigabyte G292-Z44 and G431-MM1 for 10 cards (quark2world), seen sold for about 500 to 600 USD, about 1.5k USD in offers on 2026-10-09; a Gooxi 10-GPU chassis was also linked. {{Source\|handle=quark2world\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557986225212624936}}<br />{{Source\|handle=quark2world\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557986467404451962}}<br />{{Source\|handle=quark2world\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557986644747886603}}<br />{{Source\|handle=quark2world\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557987001272246367}}<br />{{Source\|handle=quark2world\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557987057832431687}}		* Gigabyte G292-Z44 and G431-MM1 for 10 cards (quark2world), seen sold for about 500 to 600 USD, about 1.5k USD in offers on 2026-10-09; a [https://www.gooxi.us/en/ProductList/info_itemid_1696_lcid_161_pid_.html Gooxi 10-GPU chassis] was also linked. {{Source\|handle=quark2world\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557986225212624936}}<br />{{Source\|handle=quark2world\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557986467404451962}}<br />{{Source\|handle=quark2world\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557986644747886603}}<br />{{Source\|handle=quark2world\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557987001272246367}}<br />{{Source\|handle=quark2world\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557987057832431687}}
	* Supermicro SYS-1028GQ-TRT (X10DGQ board): programmerboi got three for 340 USD (about 175 USD each); PCIe 3.0 x16, dual socket, nine very loud fans (about 85 dBA). {{Source\|handle=programmerboi\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1558163867526242387}}<br />{{Source\|handle=programmerboi\|date=2026-10-18\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561406027394449560}}<br />{{Source\|handle=programmerboi\|date=2026-10-18\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561406763612504199}}<br />{{Source\|handle=programmerboi\|date=2026-10-18\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561409263664889987}}		* Supermicro SYS-1028GQ-TRT (X10DGQ board): programmerboi got three for 340 USD (about 175 USD each); PCIe 3.0 x16, dual socket, nine very loud fans (about 85 dBA). {{Source\|handle=programmerboi\|date=2026-10-09\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1558163867526242387}}<br />{{Source\|handle=programmerboi\|date=2026-10-18\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561406027394449560}}<br />{{Source\|handle=programmerboi\|date=2026-10-18\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561406763612504199}}<br />{{Source\|handle=programmerboi\|date=2026-10-18\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561409263664889987}}
	* Supermicro X12DPI-NT6 with Xeon Platinum 8375C (fedzel, CPUs only so far). {{Source\|handle=fedzel\|date=2026-10-06\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557025062689968179}}		* Supermicro X12DPI-NT6 with Xeon Platinum 8375C (fedzel, CPUs only so far). {{Source\|handle=fedzel\|date=2026-10-06\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557025062689968179}}
Line 114:		Line 114:
	* Lenovo C30 (Shudza). {{Source\|handle=shudza\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557502716337463358}}		* Lenovo C30 (Shudza). {{Source\|handle=shudza\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557502716337463358}}
	* ASRock Rack ROMED32GM-T2: endsjeremy's used board arrived with bent socket pins and neither the reseller nor ASRock Rack would repair it; nobody he found repairs SP3 sockets. {{Source\|handle=endsjeremy\|date=2026-10-08\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557576419650052157}}<br />{{Source\|handle=endsjeremy\|date=2026-10-08\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557576884051509271}}		* ASRock Rack ROMED32GM-T2: endsjeremy's used board arrived with bent socket pins and neither the reseller nor ASRock Rack would repair it; nobody he found repairs SP3 sockets. {{Source\|handle=endsjeremy\|date=2026-10-08\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557576419650052157}}<br />{{Source\|handle=endsjeremy\|date=2026-10-08\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557576884051509271}}
	* Octominer X12 Ultra mining case~~: A longer write~~-~~up by @segmond exists; see [~~https://discord.com/channels/1518634379604136178/1534370835220004885/~~1560299591319031983 Discord].~~		* An Alibaba [https://www.alibaba.com/product-detail/Custom-MIwin-DDR4-AMD-strong-Motherboard_1601593662730.html MIwin DDR4 AMD board] that endsjeremy calls a ripoff of the ROMED32GM-2T, and a [https://www.alibaba.com/product-detail/Brand-New-Stock-Single-Socket-AMD_1601635573143.html Tyan single-socket SlimSAS board] he also linked. {{Source\|handle=endsjeremy\|date=2026-10-22\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1562670871007596594}}<br />{{Source\|handle=endsjeremy\|date=2026-10-22\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1562670920886128663}}<br />{{Source\|handle=endsjeremy\|date=2026-10-22\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1562671179054063651}}<br />{{Source\|handle=endsjeremy\|date=2026-10-22\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1562671313947201556}}
			* [[Octominer X12 Ultra builds]]: segmond runs 12 cards in an Octominer X12 Ultra mining case with everything except the cards outside it. {{Source\|handle=segmond\|date=2026-10-15\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1560404429021380700}}

	== Related ==		== Related ==

ImportBot: Overview of boards, servers and hosts reported with CMP 170HX cards (Gen2, BAR1, P2P, cards per host), from Discord platform reports, PixelML/club-170hx, anoane/01-proxmox-dual-gpu-passthrough and @darkmaniac7 (Ivan) lab notes

2026-10-30T08:43:00Z

← Older revision		Revision as of 08:43, 30 October 2026
Line 20:		Line 20:
	\| Supermicro H12DSG-O-CPU (AS-4124GS-TNR), dual EPYC Rome \|\| 8 GB \|\| Gen2 x16 \|\| 64 GB \|\| yes, 504/504 byte checks including across sockets (Morrowmake P2P build) \|\| 8 \|\| {{Status\|Reported}} \|\| [[Supermicro H12DSG-O-CPU]]		\| Supermicro H12DSG-O-CPU (AS-4124GS-TNR), dual EPYC Rome \|\| 8 GB \|\| Gen2 x16 \|\| 64 GB \|\| yes, 504/504 byte checks including across sockets (Morrowmake P2P build) \|\| 8 \|\| {{Status\|Reported}} \|\| [[Supermicro H12DSG-O-CPU]]
	\|-		\|-
	\| Single-socket EPYC server, slots x8/x8, under ESXi 8 (lab) \|\| 8 GB \|\| Gen2 x8 (in guest) \|\| 64 GB with a pre-boot UEFI writer \|\| no in the VM; 12/12 pairs bare metal \|\| 4 \|\| {{Status\|Reported}} \|\| [[Virtual machines and passthrough#ESXi]]		\| Single-socket EPYC server, slots x8/x8, under ESXi 8 (lab) \|\| 8 GB \|\| Gen2 x8 (in guest) \|\| 64 GB with a pre-boot UEFI writer ([[BAR1 64 GB on ESXi]]) \|\| no in the VM; 12/12 pairs bare metal \|\| 4 \|\| {{Status\|Reported}} \|\| [[Virtual machines and passthrough#ESXi]]
	\|-		\|-
	\| Dual Broadwell Xeon with two PEX 8747 switches (PixelML club host) \|\| 8 GB \|\| Gen2 x16 \|\| 64 GB (serialized resize, kexec layout) \|\| static BAR1 P2P, 12/12 pairs at 5.79 GB/s \|\| 4 \|\| {{Status\|Reported}} \|\| [[Risers and bifurcation#What people found]]		\| Dual Broadwell Xeon with two PEX 8747 switches (PixelML club host) \|\| 8 GB \|\| Gen2 x16 \|\| 64 GB (serialized resize, kexec layout) \|\| static BAR1 P2P, 12/12 pairs at 5.79 GB/s \|\| 4 \|\| {{Status\|Reported}} \|\| [[Risers and bifurcation#What people found]]

ImportBot: Overview of boards, servers and hosts reported with CMP 170HX cards (Gen2, BAR1, P2P, cards per host), from Discord platform reports, PixelML/club-170hx, anoane/01-proxmox-dual-gpu-passthrough and @darkmaniac7 (Ivan) lab notes

2026-10-30T08:11:07Z

ImportBot: Overview of boards, servers and hosts reported with CMP 170HX cards (Gen2, BAR1, P2P, cards per host), from Discord platform reports, PixelML/club-170hx, anoane/01-proxmox-dual-gpu-passthrough and @darkmaniac7 (Ivan) lab notes

2026-10-30T08:04:54Z

New page

{{Status|Reported}} {{Cards|both}}

Which motherboards, servers and hosts people run CMP 170HX cards on, and what worked on each. Every row links to a page with the details and sources. For a step-by-step on a server, start at [[Server BIOS setup]]; for desktop and workstation boards, [[Consumer motherboards]]; for slots, risers and switches, [[Risers and bifurcation]]; for hypervisors, [[Virtual machines and passthrough]]; for distros and kernels, [[Operating systems and kernels]].

The live list of every platform report, with computed status, is [[All platforms]]. To add yours, use [[Form:Platform]].

== What the platform decides ==
* '''Gen2 link.''' The patched driver retrains the card to Gen2 from the GPU side; the board only has to not limit the slot to Gen1 and give a clean signal path. See [[Gen2 link retrain]].
* '''64 GB BAR1.''' Needs Above 4G Decoding and enough 64-bit MMIO window behind the card's bridge (64 GB + 32 MB per card, 64 GB-aligned). This is where boards differ most. See [[Resizable BAR and BAR1 64 GB]].
* '''P2P between cards.''' Needs the cards on lanes from one CPU (or the same switch), and either the 64 GB BAR1 or a P2P patch that works without it. Without P2P, engines fall back to copies through host memory: slower tensor parallel, not a failure.
* '''Cards per host.''' Limited by CPU lanes, slot spacing, power (a household outlet gives about 1600 W) and cooling. See [[Cooling and power]].
* '''Link width.''' Unmodified cards link at x4 until the [[PCIe x16 capacitor mod]]; x16 matters for tensor parallel and P2P, much less for pipeline parallel.
{{Source|handle=darkmaniac7|date=2026-10-29|note=lab notes}} {{Source|handle=wakeupmbrc1|date=2026-10-16|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1560650062474772653}} {{Source|handle=endsjeremy|date=2026-10-21|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1562585627864866888}}

== Platform table ==
"nr" = not reported. Status is the page's own status; confirmations by a second person show on each page and make it Proven there.
{| class="wikitable sortable"
! Platform !! Variant tested !! Gen2 !! BAR1 !! P2P !! Cards per host !! Status !! Page
|-
| Supermicro H12DSG-O-CPU (AS-4124GS-TNR), dual EPYC Rome || 8 GB || Gen2 x16 || 64 GB || yes (Morrowmake <code><nowiki>p2p-ecc</nowiki></code>) || 8 || {{Status|Reported}} || [[Supermicro H12DSG-O-CPU]]
|-
| Single-socket EPYC server, slots x8/x8, under ESXi 8 (lab) || 8 GB || Gen2 x8 (in guest) || 64 GB with a pre-boot UEFI writer || no in the VM; 12/12 pairs bare metal || 4 || {{Status|Reported}} || [[Virtual machines and passthrough#ESXi]]
|-
| Dual Broadwell Xeon with two PEX 8747 switches (PixelML club host) || 8 GB || Gen2 x16 || 64 GB (serialized resize, kexec layout) || static BAR1 P2P, 12/12 pairs at 5.79 GB/s || 4 || {{Status|Reported}} || [[Risers and bifurcation#What people found]]
|-
| Ryzen 9 9950X3D desktop as Proxmox host (anoane) || 8 GB || Gen2 x16 confirmed; Gen2 x1 in the documented x1 slot || nr || n/a (one CMP) || 1 CMP + 1 other GPU || {{Status|Reported}} || [[Virtual machines and passthrough#Proxmox (QEMU/KVM, VFIO)]]
|-
| Gigabyte G292-Z20 (2U, single EPYC) || nr || nr || nr || pairs on separate switches || 8 || {{Status|Reported}} + 2 confirmations || [[Gigabyte G292-Z20]]
|-
| Gigabyte G481-HA0 (dual Xeon Gold 6152) || nr || x16 links, gen nr || nr || nr || nr || {{Status|Reported}} || [[Gigabyte G481-HA0]]
|-
| Gigabyte MZ32-AR0 (single EPYC) || 8 GB || nr || nr || nr || 14 || {{Status|Reported}} || [[Gigabyte MZ32-AR0]]
|-
| Gigabyte MZ31-AR0 (single EPYC) || nr || two cards x8 by slot wiring || nr || nr || nr || {{Status|Reported}} || [[Gigabyte MZ31-AR0]]
|-
| ASRock Rack ROMED8-2T (single EPYC) || nr || nr || nr || nr || 4 || {{Status|Reported}} || [[ASRock Rack ROMED8-2T]]
|-
| Tyan S8030GM2NE + EPYC 7R13 || 10 GB || nr on this board || nr || nr || 5 x16 slots, cards behind a PLX switch || {{Status|Reported}} || [[Tyan S8030GM2NE]]
|-
| Huananzhi H12D-8D (single EPYC) || nr || nr || nr || works (Miséricorde) || 4 direct, 8+ via PLX || {{Status|Reported}} || [[Huananzhi H12D-8D]]
|-
| Supermicro SYS-4028GR-TRT2 / -TRT (dual Xeon E5) || nr || nr || nr || two PEX9797 switches, 5 slots each, all on CPU1 || 10 slots || {{Status|Reported}} + 1 confirmation || [[Supermicro SYS-4028GR-TRT2]]
|-
| Supermicro X11SPA-TF (LGA3647) || 10 GB || Gen3 (10 GB unlock) || nr || nr || up to 7 slots, shared lanes || {{Status|Reported}} || [[Supermicro X11SPA-TF]]
|-
| Hyve Edge Metal G10 (Tyan Tomcat) || nr || nr || nr || nr || x16 riser in x24 slots || {{Status|Reported}} || [[Hyve Edge Metal G10]]
|-
| Huananzhi X99-F8 / T8 || 8 GB || nr || nr || nr || 2 || {{Status|Reported}} || [[Huananzhi X99-F8]]
|-
| ASUS X99-E WS || nr || one slot never x16, one dropped to x8 || nr || nr || 4 || {{Status|Reported}} || [[ASUS X99-E WS]]
|-
| ASUS P9X79 WS || nr || nr || unlock works after an MMIOH fix || nr || 1 || {{Status|Reported}} || [[ASUS P9X79 WS]]
|-
| ASUS Pro WS WRX80E-SAGE SE WIFI || nr || nr || fails ("no space") || yes with the Morrowmake P2P patch || nr || {{Status|Reported}} || [[ASUS Pro WS WRX80E-SAGE SE WIFI]]
|-
| ASUS Pro WS WRX90E-SAGE SE || nr || nr || fails || yes with the Morrowmake P2P patch || nr || {{Status|Reported}} || [[ASUS Pro WS WRX90E-SAGE SE]]
|-
| Minisforum MS-01 + PLX 88096 || nr || nr || 64 MB with 4 cards; 64 GB on 2 of 4 after MMIOH changes || no transfers (replacedriver47); mailbox P2P (Con3po) || 4; 8 || {{Status|Reported}} || [[Minisforum MS-01]]
|-
| Dell PowerEdge R730 / Precision Rack 7910 || nr || nr || nr || nr (Proxmox VMs) || nr || {{Status|Reported}} || [[Dell PowerEdge R730]]
|-
| Dell PowerEdge R740 || nr || nr || nr || nr || nr || {{Status|Reported}} || [[Dell PowerEdge R740]]
|-
| Dell PowerEdge R720 || not tested || || || || || {{Status|Theory}} || [[Dell PowerEdge R720]]
|-
| Dell Precision T5810 || nr || nr || nr || n/a || 1 || {{Status|Reported}} || [[Dell Precision T5810]]
|-
| Dell OptiPlex 7040m (OcuLink eGPU) || nr || nr || nr || n/a || 1 || {{Status|Reported}} || [[Dell OptiPlex 7040m]]
|}

== Picks people recommend ==
* '''Single-socket EPYC (SP3)''' is the common recommendation for 4 or more cards: all lanes from one CPU, x16 slots, P2P. Named boards: Gigabyte MZ32-AR0 with a 48-core Rome such as the 7K62 (about 450 to 650 USD for board and CPU from China), ASRock Rack ROMED8-2T, Supermicro H12SSL-i, Tyan S8030GM2NE, Huananzhi H12D-8D. {{Source|handle=djpandaman|date=2026-10-12|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1559213748290592780}} {{Source|handle=wakeupmbrc1|date=2026-10-16|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1560648418433433662}} {{Source|handle=wakeupmbrc1|date=2026-10-16|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1560648485198495867}} {{Source|handle=wakeupmbrc1|date=2026-10-16|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1560652393497235579}} {{Source|handle=djpandaman|date=2026-10-22|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1562874639758131251}} SP3 gives at most 112 of the 128 lanes to slots (Arcanum). {{Source|handle=_arcanum_|date=2026-10-09|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557981458935849021}}
* '''CPU choice:''' a 3-CCX or 48-core Rome is fine and still gives full 8-channel bandwidth; the 7B13 is the cheapest 64-core Zen 3 but unnecessary for inference (Miséricorde). endsjeremy prefers a Rome with the full L3 cache (7742, 7532). {{Source|handle=endsjeremy|date=2026-10-18|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561404368908058768}} {{Source|handle=wakeupmbrc1|date=2026-10-18|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561404732763799553}} {{Source|handle=endsjeremy|date=2026-10-18|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561405132871176285}} {{Source|handle=wakeupmbrc1|date=2026-10-18|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561405221119459449}}
* '''GPU servers''' (Gigabyte G292-Z20, Supermicro SYS-4028GR-TRT2, Gigabyte G481-HA0) solve power, risers and cooling for 8 to 10 cards, at the cost of noise.
* '''For 8 GB cards''' Gen3 platforms are enough, since the cards link at Gen2; dual Xeon v3/v4 Supermicro X10 boards with 4 to 6 slots work fine (djpandaman, Miséricorde). {{Source|handle=wakeupmbrc1|date=2026-10-12|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1559217419648630899}} {{Source|handle=wakeupmbrc1|date=2026-10-12|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1559217454113361973}} {{Source|handle=djpandaman|date=2026-10-12|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1559217525881839627}} {{Source|handle=djpandaman|date=2026-10-12|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1559217615296270497}}
* '''Two cards cheaply:''' a Huananzhi X99 board. {{Source|handle=wakeupmbrc1|date=2026-10-07|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557502108347928577}}
* '''Any PC plus a 10-port PEX88096 switch''' (Gen4 or newer CPU for full bandwidth). {{Source|handle=djpandaman|date=2026-10-22|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1562874639758131251}}
* '''All lanes from one CPU''' or P2P will not work properly (Miséricorde). {{Source|handle=wakeupmbrc1|date=2026-10-16|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1560650062474772653}} Whether P2P works across sockets is disputed; see [[Risers and bifurcation#Topology rules of thumb]]. {{Status|Contested}}

== Host CPU and RAM ==
* '''RAM for offload.''' 1a2b3c0967: 256 GB of system RAM is not overkill, because n-gram (engram/PLE) offloading is getting popular with newer models. {{Source|handle=1a2b3c0967|date=2026-10-19|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561876104061386763}} Miséricorde: PLE lookups are latency bound, so RAM speed hardly matters (even DDR3 works), but RAM versus NVMe does. {{Status|Theory}} {{Source|handle=wakeupmbrc1|date=2026-10-17|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1560991110677930017}} Details in [[Recipes]].
* '''Strata-style engines on old hosts:''' Strata decode did not seem to scale with system memory speed, which makes DDR3 and DDR4 Xeon hosts (for example an E5-2696 v3 with DDR3) candidates; Miséricorde asked for a test with CPU cores disabled to check whether it is compute bound instead, and mrpandagod wants to try Optane PMem 200. {{Status|Theory}} {{Source|handle=wakeupmbrc1|date=2026-10-07|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557262803969253437}} {{Source|handle=djpandaman|date=2026-10-07|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557263101647257674}} {{Source|handle=wakeupmbrc1|date=2026-10-07|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557263355150864505}} {{Source|handle=wakeupmbrc1|date=2026-10-08|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557836716021129357}}
* '''Cheap ECC DDR4:''' Micron NVDIMM-N modules (RDIMMs with flash backup) sell below plain RDIMMs because support is spotty; UnlikelyPotato runs them as normal ECC DDR4 at 2666 on a Supermicro X11DAi-N with Xeon 8173 CPUs. They should enumerate as plain ECC DDR4 on boards without NVDIMM support, but that is not guaranteed. {{Source|handle=unlikelypotato|date=2026-10-08|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557841010732699698}} {{Source|handle=unlikelypotato|date=2026-10-08|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557841098393784372}} {{Source|handle=unlikelypotato|date=2026-10-08|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557869702959857744}} {{Source|handle=unlikelypotato|date=2026-10-08|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557870602881343552}} {{Source|handle=unlikelypotato|date=2026-10-08|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557871076611203154}}
* '''Do RDIMMs need to match?''' {{Status|Contested}}
** Miséricorde: RDIMMs do not need to be matched; you can even mix ICs (he bought 11 used 16 GB sticks at 40 USD each for an EPYC Milan build). {{Source|handle=wakeupmbrc1|date=2026-10-17|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813031363973225}} {{Source|handle=wakeupmbrc1|date=2026-10-17|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813484072247297}} {{Source|handle=wakeupmbrc1|date=2026-10-17|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813506113306756}}
** endsjeremy: he insists on type and revision matching, especially with used dual-rank memory. {{Source|handle=endsjeremy|date=2026-10-17|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813660652314675}}
* '''CPU cooler:''' GlobalApathy and Dc both use the Arctic SP3 cooler on their EPYC hosts; Dc: it keeps the CPU cool and is silent. {{Source|handle=globalapathy|date=2026-10-11|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1558682489017335870}} {{Source|handle=daveedcoles|date=2026-10-11|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1558694978979500043}}
* '''Renting the box out:''' endsjeremy suggests about 2:1 system RAM to VRAM and more than 16 cores for rentals. {{Source|handle=endsjeremy|date=2026-10-07|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557504597457903700}} {{Source|handle=endsjeremy|date=2026-10-07|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557504862894424179}}
* '''Broadwell CPUs''' may halve some Qwen Flash recipe numbers (ElectroHobo); segmond attributes the same gap to no x16 and no P2P. {{Status|Theory}} {{Source|handle=segmond|date=2026-10-20|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1561950054254645291}} {{Source|handle=nathang6866|date=2026-10-20|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1561951080882114701}}
* '''DDR3 hosts''' for RAM-heavy builds (X99 with DDR3-capable Xeons, old LGA2011 servers that train 64 GB DDR3 LRDIMMs) are cheaper per GB; see [[Consumer motherboards#X99 and X79 (including Huananzhi)]].
* '''Quad-socket Supermicro SYS-8048B-TRFT / SYS-4048B-TRFT:''' up to 3 TB RAM and 96 cores, about 300 W idle and 1000 W peak (endsjeremy); NUMA and IOMMU gotchas. {{Status|Theory}} {{Source|handle=endsjeremy|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1564029995243016223}} {{Source|handle=endsjeremy|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1564030127635963907}}

== Host idle power ==
Host idle power can match the cards'. fnerdman's four cards on a ROMED8-2T idle at about the same wattage as the server CPU and RAM. {{Source|handle=fnerdman|date=2026-10-26|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1564244395887435847}} fafinet cannot get his EPYC 7502 host (one 4 GB RDIMM, AST2500 BMC daughterboard) below 120 W at the wall; his Gigabyte MC62-G40 with a Threadripper Pro 3995WX idles at 300 W. {{Source|handle=fafinet|date=2026-10-26|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1564263781108940871}} Where idle power goes on EPYC and Threadripper is disputed: {{Status|Contested}}
* alanoo: RAM and its controllers alone idle near 50 W, about 80 W for 8 channels on EPYC. {{Source|handle=alanoo|date=2026-10-26|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1564282354455814205}}
* gnasherrr: the IO die basically never idles and is the major draw. {{Source|handle=gnasherrr|date=2026-10-26|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1564312835935305749}}
* _L_: Zen 3 Threadripper can have very low IO die power, under 20 W SOC idle on a 5995WX at JEDEC memory settings, but RAM overclocking ruins idle. {{Source|handle=lia_oc|date=2026-10-26|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1564313953444692011}} {{Source|handle=lia_oc|date=2026-10-26|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1564314222412955852}}
See [[Cooling and power]].

== Large rigs ==
RH-3D is cutting a 32-card rig to 16: two 16-card servers are easier to maintain than one with 32. {{Source|handle=rh3dai|date=2026-10-14|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1560061063834247178}} BackwardsMiner ran 45 GPUs for a day on an Intel 11th-gen CPU until the BIOS ran out of PCIe resources; his 32-card build used an Intel NUC 9 Extreme as the host over PLX switches and is now a 16-card rig to cut heat and power. {{Source|handle=backwardsminer|date=2026-10-15|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1560212119599718432}} {{Source|handle=backwardsminer|date=2026-10-15|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1560216455084249191}} {{Source|handle=backwardsminer|date=2026-10-15|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1560218164573372467}}

== Mixing with other GPUs ==
Other NVIDIA cards in the same host must be Turing or newer (same driver branch); see [[Operating systems and kernels#Mixed GPUs in one host]]. Pooling a CMP with AMD and other NVIDIA cards in one vLLM model worked for Con3po but ran very poorly. {{Source|handle=con3po7016|date=2026-10-15|link=https://discord.com/channels/1518634379604136178/1518695289278566450/1560090508292919357}} Mixing sm70 V100s with sm80 CMPs is an untested idea ("driver nightmare", kellertreppe123). {{Status|Theory}} {{Source|handle=kellertreppe123|date=2026-10-24|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563603462552293497}} {{Source|handle=kellertreppe123|date=2026-10-24|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1563603652256731312}}

== Suggested, no CMP result yet ==
These come up as hosts but nobody has posted a CMP result on them. Good candidates for testing tasks on the board.
* Gigabyte G292-Z44 and G431-MM1 for 10 cards (quark2world), seen sold for about 500 to 600 USD, about 1.5k USD in offers on 2026-10-09; a Gooxi 10-GPU chassis was also linked. {{Source|handle=quark2world|date=2026-10-09|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557986225212624936}} {{Source|handle=quark2world|date=2026-10-09|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557986467404451962}} {{Source|handle=quark2world|date=2026-10-09|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557986644747886603}} {{Source|handle=quark2world|date=2026-10-09|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557987001272246367}} {{Source|handle=quark2world|date=2026-10-09|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557987057832431687}}
* Supermicro SYS-1028GQ-TRT (X10DGQ board): programmerboi got three for 340 USD (about 175 USD each); PCIe 3.0 x16, dual socket, nine very loud fans (about 85 dBA). {{Source|handle=programmerboi|date=2026-10-09|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1558163867526242387}} {{Source|handle=programmerboi|date=2026-10-18|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561406027394449560}} {{Source|handle=programmerboi|date=2026-10-18|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561406763612504199}} {{Source|handle=programmerboi|date=2026-10-18|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561409263664889987}}
* Supermicro X12DPI-NT6 with Xeon Platinum 8375C (fedzel, CPUs only so far). {{Source|handle=fedzel|date=2026-10-06|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1557025062689968179}}
* Dell Precision T7600 (djpandaman). {{Source|handle=djpandaman|date=2026-10-09|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557968463459520543}}
* Lenovo C30 (Shudza). {{Source|handle=shudza|date=2026-10-07|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557502716337463358}}
* ASRock Rack ROMED32GM-T2: endsjeremy's used board arrived with bent socket pins and neither the reseller nor ASRock Rack would repair it; nobody he found repairs SP3 sockets. {{Source|handle=endsjeremy|date=2026-10-08|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557576419650052157}} {{Source|handle=endsjeremy|date=2026-10-08|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557576884051509271}}
* Octominer X12 Ultra mining case: A longer write-up by @segmond exists; see [https://discord.com/channels/1518634379604136178/1534370835220004885/1560299591319031983 Discord].

== Related ==
[[Server BIOS setup]], [[Consumer motherboards]], [[Risers and bifurcation]], [[Virtual machines and passthrough]], [[Operating systems and kernels]], [[Unlocks]], [[Gen2 link retrain]], [[Resizable BAR and BAR1 64 GB]], [[Troubleshooting]], [[Card not detected or BAR allocation fails]], [[Cooling and power]], [[Chassis builds]], [[Power supplies and cabling]], [[P2P]], [[Which card do I have]], [[ASRock PG-1600G]] (a PSU, not a board).

{{ImportFooter}}
[[Category:Platforms]]
[[Category:8 GB]]
[[Category:10 GB]]

← Older revision		Revision as of 10:02, 30 October 2026
Line 87:		Line 87:
	** Miséricorde: RDIMMs do not need to be matched; you can even mix ICs (he bought 11 used 16 GB sticks at 40 USD each for an EPYC Milan build). {{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813031363973225}}<br />{{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813484072247297}}<br />{{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813506113306756}}		** Miséricorde: RDIMMs do not need to be matched; you can even mix ICs (he bought 11 used 16 GB sticks at 40 USD each for an EPYC Milan build). {{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813031363973225}}<br />{{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813484072247297}}<br />{{Source\|handle=wakeupmbrc1\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813506113306756}}
	** endsjeremy: he insists on type and revision matching, especially with used dual-rank memory. {{Source\|handle=endsjeremy\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813660652314675}}		** endsjeremy: he insists on type and revision matching, especially with used dual-rank memory. {{Source\|handle=endsjeremy\|date=2026-10-17\|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1560813660652314675}}
	* '''CPU cooler:''' GlobalApathy ~~and Dc both use~~ the Arctic SP3 cooler ~~on their EPYC hosts~~; Dc: it keeps the CPU cool and is silent. {{Source\|handle=globalapathy\|date=2026-10-11\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1558682489017335870}}<br />{{Source\|handle=daveedcoles\|date=2026-10-11\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1558694978979500043}}		* '''CPU cooler:''' GlobalApathy rates the Arctic cooler for SP3 (EPYC) as pretty good. He recognised the same Arctic cooler in Dc's build photo; Dc did not name the model, and says it keeps the CPU cool and is silent. {{Source\|handle=globalapathy\|date=2026-10-11\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1558682489017335870}}<br />{{Source\|handle=daveedcoles\|date=2026-10-11\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1558694978979500043}}<br />{{Source\|handle=globalapathy\|date=2026-10-14\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1559752062852669461}}
	* '''Renting the box out:''' endsjeremy suggests about 2:1 system RAM to VRAM and more than 16 cores for rentals. {{Source\|handle=endsjeremy\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557504597457903700}}<br />{{Source\|handle=endsjeremy\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557504862894424179}}		* '''Renting the box out:''' endsjeremy suggests about 2:1 system RAM to VRAM and more than 16 cores for rentals. {{Source\|handle=endsjeremy\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557504597457903700}}<br />{{Source\|handle=endsjeremy\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557504862894424179}}
	* '''Broadwell CPUs''' may halve some Qwen Flash recipe numbers (ElectroHobo); segmond attributes the same gap to no x16 and no P2P. {{Status\|Theory}} {{Source\|handle=segmond\|date=2026-10-20\|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1561950054254645291}}<br />{{Source\|handle=nathang6866\|date=2026-10-20\|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1561951080882114701}}		* '''Broadwell CPUs''' may halve some Qwen Flash recipe numbers (ElectroHobo); segmond attributes the same gap to no x16 and no P2P. {{Status\|Theory}} {{Source\|handle=segmond\|date=2026-10-20\|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1561950054254645291}}<br />{{Source\|handle=nathang6866\|date=2026-10-20\|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1561951080882114701}}

← Older revision		Revision as of 08:11, 30 October 2026
Line 8:		Line 8:
	* '''Gen2 link.''' The patched driver retrains the card to Gen2 from the GPU side; the board only has to not limit the slot to Gen1 and give a clean signal path. See [[Gen2 link retrain]].		* '''Gen2 link.''' The patched driver retrains the card to Gen2 from the GPU side; the board only has to not limit the slot to Gen1 and give a clean signal path. See [[Gen2 link retrain]].
	* '''64 GB BAR1.''' Needs Above 4G Decoding and enough 64-bit MMIO window behind the card's bridge (64 GB + 32 MB per card, 64 GB-aligned). This is where boards differ most. See [[Resizable BAR and BAR1 64 GB]].		* '''64 GB BAR1.''' Needs Above 4G Decoding and enough 64-bit MMIO window behind the card's bridge (64 GB + 32 MB per card, 64 GB-aligned). This is where boards differ most. See [[Resizable BAR and BAR1 64 GB]].
	* '''P2P between cards.''' Needs the ~~cards on lanes from one CPU (or the same switch),~~ and either the 64 GB BAR1 or a P2P patch that works without it. Without P2P, engines fall back to copies through host memory: slower tensor parallel, not a failure.		* '''P2P between cards.''' Needs a P2P build of the unlock and either the 64 GB BAR1 or a P2P patch that works without it. Cards on one CPU or one switch is the usual layout; a dual-socket EPYC server also passed across sockets. Without P2P, engines fall back to copies through host memory: slower tensor parallel, not a failure.
	* '''Cards per host.''' Limited by CPU lanes, slot spacing, power (a household outlet gives about 1600 W) and cooling. See [[Cooling and power]].		* '''Cards per host.''' Limited by CPU lanes, slot spacing, power (a household outlet gives about 1600 W) and cooling. See [[Cooling and power]].
	* '''Link width.''' Unmodified cards link at x4 until the [[PCIe x16 capacitor mod]]; x16 matters for tensor parallel and P2P, much less for pipeline parallel.		* '''Link width.''' Unmodified cards link at x4 until the [[PCIe x16 capacitor mod]]; x16 matters for tensor parallel and P2P, much less for pipeline parallel.
Line 18:		Line 18:
	! Platform !! Variant tested !! Gen2 !! BAR1 !! P2P !! Cards per host !! Status !! Page		! Platform !! Variant tested !! Gen2 !! BAR1 !! P2P !! Cards per host !! Status !! Page
	\|-		\|-
	\| Supermicro H12DSG-O-CPU (AS-4124GS-TNR), dual EPYC Rome \|\| 8 GB \|\| Gen2 x16 \|\| 64 GB \|\| yes (Morrowmake ~~<code><nowiki>p2p-ecc</nowiki></code>~~) \|\| 8 \|\| {{Status\|Reported}} \|\| [[Supermicro H12DSG-O-CPU]]		\| Supermicro H12DSG-O-CPU (AS-4124GS-TNR), dual EPYC Rome \|\| 8 GB \|\| Gen2 x16 \|\| 64 GB \|\| yes, 504/504 byte checks including across sockets (Morrowmake P2P build) \|\| 8 \|\| {{Status\|Reported}} \|\| [[Supermicro H12DSG-O-CPU]]
	\|-		\|-
	\| Single-socket EPYC server, slots x8/x8, under ESXi 8 (lab) \|\| 8 GB \|\| Gen2 x8 (in guest) \|\| 64 GB with a pre-boot UEFI writer \|\| no in the VM; 12/12 pairs bare metal \|\| 4 \|\| {{Status\|Reported}} \|\| [[Virtual machines and passthrough#ESXi]]		\| Single-socket EPYC server, slots x8/x8, under ESXi 8 (lab) \|\| 8 GB \|\| Gen2 x8 (in guest) \|\| 64 GB with a pre-boot UEFI writer \|\| no in the VM; 12/12 pairs bare metal \|\| 4 \|\| {{Status\|Reported}} \|\| [[Virtual machines and passthrough#ESXi]]
Line 76:		Line 76:
	* '''Two cards cheaply:''' a Huananzhi X99 board. {{Source\|handle=wakeupmbrc1\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557502108347928577}}		* '''Two cards cheaply:''' a Huananzhi X99 board. {{Source\|handle=wakeupmbrc1\|date=2026-10-07\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557502108347928577}}
	* '''Any PC plus a 10-port PEX88096 switch''' (Gen4 or newer CPU for full bandwidth). {{Source\|handle=djpandaman\|date=2026-10-22\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1562874639758131251}}		* '''Any PC plus a 10-port PEX88096 switch''' (Gen4 or newer CPU for full bandwidth). {{Source\|handle=djpandaman\|date=2026-10-22\|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1562874639758131251}}
	* '''~~All lanes~~ from one CPU''' or P2P ~~will not work properly~~ (Miséricorde). {{Source\|handle=wakeupmbrc1\|date=2026-10-16\|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1560650062474772653}} ~~Whether~~ P2P ~~works~~ across sockets ~~is disputed; see~~ [[~~Risers and bifurcation#Topology rules of thumb~~]]. {{~~Status~~\|~~Contested~~}}		* '''Lanes from one CPU''' is the usual advice for P2P (Miséricorde). {{Source\|handle=wakeupmbrc1\|date=2026-10-16\|link=https://discord.com/channels/1518634379604136178/1528826330454429706/1560650062474772653}}
			* '''P2P across sockets works on a dual-socket EPYC server:''' all 504 <code><nowiki>p2p-content-check.py</nowiki></code> checks passed on 8 cards spread over both CPUs of a Supermicro H12DSG-O-CPU, including the cross-socket pairs. {{Status\|Proven}} {{Source\|handle=darkmaniac7\|date=2026-10-29\|note=lab notes}} See [[Supermicro H12DSG-O-CPU]].
			** Conflicting earlier source: Miséricorde said P2P does not work across sockets. {{Source\|handle=wakeupmbrc1\|date=2026-10-18\|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561407364622458943}}

	== Host CPU and RAM ==		== Host CPU and RAM ==