Jump to content
Toggle search
Toggle menu
CMP Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Platforms

From CMP Wiki
More actions

Reported 8 GB and 10 GB

Which motherboards, servers and hosts people run CMP 170HX cards on, and what worked on each. Every row links to a page with the details and sources. For a step-by-step on a server, start at Server BIOS setup; for desktop and workstation boards, Consumer motherboards; for slots, risers and switches, Risers and bifurcation; for hypervisors, Virtual machines and passthrough; for distros and kernels, Operating systems and kernels.

The live list of every platform report, with computed status, is All platforms. To add yours, use Form:Platform.

What the platform decides

  • Gen2 link. The patched driver retrains the card to Gen2 from the GPU side; the board only has to not limit the slot to Gen1 and give a clean signal path. See Gen2 link retrain.
  • 64 GB BAR1. Needs Above 4G Decoding and enough 64-bit MMIO window behind the card's bridge (64 GB + 32 MB per card, 64 GB-aligned). This is where boards differ most. See Resizable BAR and BAR1 64 GB.
  • P2P between cards. Needs a P2P build of the unlock and either the 64 GB BAR1 or a P2P patch that works without it. Cards on one CPU or one switch is the usual layout; a dual-socket EPYC server also passed across sockets. Without P2P, engines fall back to copies through host memory: slower tensor parallel, not a failure.
  • Cards per host. Limited by CPU lanes, slot spacing, power (a household outlet gives about 1600 W) and cooling. See Cooling and power.
  • Link width. Unmodified cards link at x4 until the PCIe x16 capacitor mod; x16 matters for tensor parallel and P2P, much less for pipeline parallel.

Source: @darkmaniac7, 2026-10-29 (lab notes) Source: @wakeupmbrc1, 2026-10-16
Source: @endsjeremy, 2026-10-21

Platform table

"nr" = not reported. Status is the page's own status; confirmations by a second person show on each page and make it Proven there.

Platform Variant tested Gen2 BAR1 P2P Cards per host Status Page
Supermicro H12DSG-O-CPU (AS-4124GS-TNR), dual EPYC Rome 8 GB Gen2 x16 64 GB yes, 504/504 byte checks including across sockets (Morrowmake P2P build) 8 Reported Supermicro H12DSG-O-CPU
Single-socket EPYC server, slots x8/x8, under ESXi 8 (lab) 8 GB Gen2 x8 (in guest) 64 GB with a pre-boot UEFI writer (BAR1 64 GB on ESXi) no in the VM; 12/12 pairs bare metal 4 Reported Virtual machines and passthrough#ESXi
Dual Broadwell Xeon with two PEX 8747 switches (PixelML club host) 8 GB Gen2 x16 64 GB (serialized resize, kexec layout) static BAR1 P2P, 12/12 pairs at 5.79 GB/s 4 Reported Risers and bifurcation#What people found
Ryzen 9 9950X3D desktop as Proxmox host (anoane) 8 GB Gen2 x16 confirmed; Gen2 x1 in the documented x1 slot nr n/a (one CMP) 1 CMP + 1 other GPU Reported Virtual machines and passthrough#Proxmox (QEMU/KVM, VFIO)
Gigabyte G292-Z20 (2U, single EPYC) nr nr nr pairs on separate switches 8 Reported + 2 confirmations Gigabyte G292-Z20
Gigabyte G481-HA0 (dual Xeon Gold 6152) nr x16 links, gen nr nr nr nr Reported Gigabyte G481-HA0
Gigabyte MZ32-AR0 (single EPYC) 8 GB nr nr nr 14 Reported Gigabyte MZ32-AR0
Gigabyte MZ31-AR0 (single EPYC) nr two cards x8 by slot wiring nr nr nr Reported Gigabyte MZ31-AR0
ASRock Rack ROMED8-2T (single EPYC) nr nr nr nr 4 Reported ASRock Rack ROMED8-2T
Tyan S8030GM2NE + EPYC 7R13 10 GB nr on this board nr nr 5 x16 slots, cards behind a PLX switch Reported Tyan S8030GM2NE
Huananzhi H12D-8D (single EPYC) nr nr nr works (Miséricorde) 4 direct, 8+ via PLX Reported Huananzhi H12D-8D
Supermicro SYS-4028GR-TRT2 / -TRT (dual Xeon E5) nr nr nr two PEX9797 switches, 5 slots each, all on CPU1 10 slots Reported + 1 confirmation Supermicro SYS-4028GR-TRT2
Supermicro X11SPA-TF (LGA3647) 10 GB Gen3 (10 GB unlock) nr nr up to 7 slots, shared lanes Reported Supermicro X11SPA-TF
Hyve Edge Metal G10 (Tyan Tomcat) nr nr nr nr x16 riser in x24 slots Reported Hyve Edge Metal G10
Huananzhi X99-F8 / T8 8 GB nr nr nr 2 Reported Huananzhi X99-F8
ASUS X99-E WS nr one slot never x16, one dropped to x8 nr nr 4 Reported ASUS X99-E WS
ASUS P9X79 WS nr nr unlock works after an MMIOH fix nr 1 Reported ASUS P9X79 WS
ASUS Pro WS WRX80E-SAGE SE WIFI nr nr fails ("no space") yes with the Morrowmake P2P patch nr Reported ASUS Pro WS WRX80E-SAGE SE WIFI
ASUS Pro WS WRX90E-SAGE SE nr nr fails yes with the Morrowmake P2P patch nr Reported ASUS Pro WS WRX90E-SAGE SE
Minisforum MS-01 + PLX 88096 nr nr 64 MB with 4 cards; 64 GB on 2 of 4 after MMIOH changes no transfers (replacedriver47); mailbox P2P (Con3po) 4; 8 Reported Minisforum MS-01
Dell PowerEdge R730 / Precision Rack 7910 nr nr nr nr (Proxmox VMs) nr Reported Dell PowerEdge R730
Dell PowerEdge R740 nr nr nr nr nr Reported Dell PowerEdge R740
Dell PowerEdge R720 not tested Theory Dell PowerEdge R720
Dell Precision T5810 nr nr nr n/a 1 Reported Dell Precision T5810
Dell OptiPlex 7040m (OcuLink eGPU) nr nr nr n/a 1 Reported Dell OptiPlex 7040m

Picks people recommend

  • Single-socket EPYC (SP3) is the common recommendation for 4 or more cards: all lanes from one CPU, x16 slots, P2P. Named boards: Gigabyte MZ32-AR0 with a 48-core Rome such as the 7K62 (about 450 to 650 USD for board and CPU from China), ASRock Rack ROMED8-2T, Supermicro H12SSL-i, Tyan S8030GM2NE, Huananzhi H12D-8D. Source: @djpandaman, 2026-10-12
    Source: @wakeupmbrc1, 2026-10-16
    Source: @wakeupmbrc1, 2026-10-16
    Source: @wakeupmbrc1, 2026-10-16
    Source: @djpandaman, 2026-10-22 SP3 gives at most 112 of the 128 lanes to slots (Arcanum). Source: @_arcanum_, 2026-10-09
  • CPU choice: a 3-CCX or 48-core Rome is fine and still gives full 8-channel bandwidth; the 7B13 is the cheapest 64-core Zen 3 but unnecessary for inference (Miséricorde). endsjeremy prefers a Rome with the full L3 cache (7742, 7532). Source: @endsjeremy, 2026-10-18
    Source: @wakeupmbrc1, 2026-10-18
    Source: @endsjeremy, 2026-10-18
    Source: @wakeupmbrc1, 2026-10-18
  • GPU servers (Gigabyte G292-Z20, Supermicro SYS-4028GR-TRT2, Gigabyte G481-HA0) solve power, risers and cooling for 8 to 10 cards, at the cost of noise.
  • For 8 GB cards Gen3 platforms are enough, since the cards link at Gen2; dual Xeon v3/v4 Supermicro X10 boards with 4 to 6 slots work fine (djpandaman, Miséricorde). Source: @wakeupmbrc1, 2026-10-12
    Source: @wakeupmbrc1, 2026-10-12
    Source: @djpandaman, 2026-10-12
    Source: @djpandaman, 2026-10-12
  • Two cards cheaply: a Huananzhi X99 board. Source: @wakeupmbrc1, 2026-10-07
  • Any PC plus a 10-port PEX88096 switch (Gen4 or newer CPU for full bandwidth). Source: @djpandaman, 2026-10-22
  • Lanes from one CPU is the usual advice for P2P (Miséricorde). Source: @wakeupmbrc1, 2026-10-16
  • P2P across sockets works on a dual-socket EPYC server: all 504 p2p-content-check.py checks passed on 8 cards spread over both CPUs of a Supermicro H12DSG-O-CPU, including the cross-socket pairs. Proven Source: @darkmaniac7, 2026-10-29 (lab notes) See Supermicro H12DSG-O-CPU.
    • Conflicting earlier source: Miséricorde said P2P does not work across sockets. Source: @wakeupmbrc1, 2026-10-18

Host CPU and RAM

  • RAM for offload. 1a2b3c0967: 256 GB of system RAM is not overkill, because n-gram (engram/PLE) offloading is getting popular with newer models. Source: @1a2b3c0967, 2026-10-19 Miséricorde: PLE lookups are latency bound, so RAM speed hardly matters (even DDR3 works), but RAM versus NVMe does. Theory Source: @wakeupmbrc1, 2026-10-17 Details in Recipes.
  • Strata-style engines on old hosts: Strata decode did not seem to scale with system memory speed, which makes DDR3 and DDR4 Xeon hosts (for example an E5-2696 v3 with DDR3) candidates; Miséricorde asked for a test with CPU cores disabled to check whether it is compute bound instead, and mrpandagod wants to try Optane PMem 200. Theory Source: @wakeupmbrc1, 2026-10-07
    Source: @djpandaman, 2026-10-07
    Source: @wakeupmbrc1, 2026-10-07
    Source: @wakeupmbrc1, 2026-10-08
  • Cheap ECC DDR4: Micron NVDIMM-N modules (RDIMMs with flash backup) sell below plain RDIMMs because support is spotty; UnlikelyPotato runs them as normal ECC DDR4 at 2666 on a Supermicro X11DAi-N with Xeon 8173 CPUs. They should enumerate as plain ECC DDR4 on boards without NVDIMM support, but that is not guaranteed. Source: @unlikelypotato, 2026-10-08
    Source: @unlikelypotato, 2026-10-08
    Source: @unlikelypotato, 2026-10-08
    Source: @unlikelypotato, 2026-10-08
    Source: @unlikelypotato, 2026-10-08
  • Do RDIMMs need to match? Contested
    • Miséricorde: RDIMMs do not need to be matched; you can even mix ICs (he bought 11 used 16 GB sticks at 40 USD each for an EPYC Milan build). Source: @wakeupmbrc1, 2026-10-17
      Source: @wakeupmbrc1, 2026-10-17
      Source: @wakeupmbrc1, 2026-10-17
    • endsjeremy: he insists on type and revision matching, especially with used dual-rank memory. Source: @endsjeremy, 2026-10-17
  • CPU cooler: GlobalApathy rates the Arctic cooler for SP3 (EPYC) as pretty good. He recognised the same Arctic cooler in Dc's build photo; Dc did not name the model, and says it keeps the CPU cool and is silent. Source: @globalapathy, 2026-10-11
    Source: @daveedcoles, 2026-10-11
    Source: @globalapathy, 2026-10-14
  • Renting the box out: endsjeremy suggests about 2:1 system RAM to VRAM and more than 16 cores for rentals. Source: @endsjeremy, 2026-10-07
    Source: @endsjeremy, 2026-10-07
  • Broadwell CPUs may halve some Qwen Flash recipe numbers (ElectroHobo); segmond attributes the same gap to no x16 and no P2P. Theory Source: @segmond, 2026-10-20
    Source: @nathang6866, 2026-10-20
  • DDR3 hosts for RAM-heavy builds (X99 with DDR3-capable Xeons, old LGA2011 servers that train 64 GB DDR3 LRDIMMs) are cheaper per GB; see Consumer motherboards#X99 and X79 (including Huananzhi).
  • Quad-socket Supermicro SYS-8048B-TRFT / SYS-4048B-TRFT: up to 3 TB RAM and 96 cores, about 300 W idle and 1000 W peak (endsjeremy); NUMA and IOMMU gotchas. Theory Source: @endsjeremy, 2026-10-25
    Source: @endsjeremy, 2026-10-25

Host idle power

Host idle power can match the cards'. fnerdman's four cards on a ROMED8-2T idle at about the same wattage as the server CPU and RAM. Source: @fnerdman, 2026-10-26 fafinet cannot get his EPYC 7502 host (one 4 GB RDIMM, AST2500 BMC daughterboard) below 120 W at the wall; his Gigabyte MC62-G40 with a Threadripper Pro 3995WX idles at 300 W. Source: @fafinet, 2026-10-26 Where idle power goes on EPYC and Threadripper is disputed: Contested

  • alanoo: RAM and its controllers alone idle near 50 W, about 80 W for 8 channels on EPYC. Source: @alanoo, 2026-10-26
  • gnasherrr: the IO die basically never idles and is the major draw. Source: @gnasherrr, 2026-10-26
  • _L_: Zen 3 Threadripper can have very low IO die power, under 20 W SOC idle on a 5995WX at JEDEC memory settings, but RAM overclocking ruins idle. Source: @lia_oc, 2026-10-26
    Source: @lia_oc, 2026-10-26

See Cooling and power.

Large rigs

RH-3D is cutting a 32-card rig to 16: two 16-card servers are easier to maintain than one with 32. Source: @rh3dai, 2026-10-14 BackwardsMiner ran 45 GPUs for a day on an Intel 11th-gen CPU until the BIOS ran out of PCIe resources; his 32-card build used an Intel NUC 9 Extreme as the host over PLX switches and is now a 16-card rig to cut heat and power. Source: @backwardsminer, 2026-10-15
Source: @backwardsminer, 2026-10-15
Source: @backwardsminer, 2026-10-15

Mixing with other GPUs

Other NVIDIA cards in the same host must be Turing or newer (same driver branch); see Operating systems and kernels#Mixed GPUs in one host. Pooling a CMP with AMD and other NVIDIA cards in one vLLM model worked for Con3po but ran very poorly. Source: @con3po7016, 2026-10-15 Mixing sm70 V100s with sm80 CMPs is an untested idea ("driver nightmare", kellertreppe123). Theory Source: @kellertreppe123, 2026-10-24
Source: @kellertreppe123, 2026-10-24

Suggested, no CMP result yet

These come up as hosts but nobody has posted a CMP result on them. Good candidates for testing tasks on the board.

Related

Server BIOS setup, Consumer motherboards, Risers and bifurcation, Virtual machines and passthrough, Operating systems and kernels, Unlocks, Gen2 link retrain, Resizable BAR and BAR1 64 GB, Troubleshooting, Card not detected or BAR allocation fails, Cooling and power, Chassis builds, Power supplies and cabling, P2P, Which card do I have, ASRock PG-1600G (a PSU, not a board).

Imported from the cmpunlocker Discord with credit; see Credits.
Retrieved from "https://cmp.wiki/w/index.php?title=Platforms&oldid=468"