https://cmp.wiki/w/index.php?action=history&feed=atom&title=Supermicro_SYS-4028GR-TRT2 2026-10-31T08:49:36Z Revision history for this page on the wiki MediaWiki 1.43.11 https://cmp.wiki/w/index.php?title=Supermicro_SYS-4028GR-TRT2&diff=202&oldid=prev 2026-10-30T08:05:05Z

<p>Supermicro SYS-4028GR-TRT2 / 4028GR-TRT 10-slot GPU server topology for CMP 170HX, from Discord (markshowa, ElectroHobo, Kaatchi, endsjeremy); sources: @markshowa, @endsjeremy, @nathang6866, @catchyuser</p> <p><b>New page</b></p><div>{{Platform<br /> |board=Supermicro SYS-4028GR-TRT2 (X10DRG-O-PCIE class board; also 4028GR-TRT)<br /> |cpu=Dual Intel Xeon E5 v3/v4 (E5-2673 v4 used), DDR4 up to 2400 MHz<br /> |bios_version=<br /> |settings=<br /> |gen2_result=<br /> |rebar_result=<br /> |p2p_result=All 10 GPU slots on CPU1 behind two PEX9797 (PCIe 3.0) switches, five slots each: in-switch P2P expected good, cross-switch traffic goes through the CPU<br /> |os_driver=<br /> |virtualization=Bare metal<br /> |cards=<br /> |status=Reported<br /> |tested_by=markshowa<br /> |source=https://discord.com/channels/1518634379604136178/1534370835220004885/1561798498905755670<br /> }}<br /> <br /> A 4U dual-socket GPU server with 10 GPU slots, all on CPU1 behind two PCIe 3.0 switches.<br /> <br /> == Topology ==<br /> markshowa: the TRT2 has a single root; all 10 GPU slots are on CPU1 through two PEX9797 switches, each on one PCIe 3.0 x16 uplink. That caps model load and unload at about 16 GB/s, and GPU traffic across the two switches goes through the CPU. He plans two groups of four cards and expects TP4 x PP2 to work well. {{Source|handle=markshowa|date=2026-10-17|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561012306987716711}}&lt;br /&gt;{{Source|handle=markshowa|date=2026-10-19|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1561798498905755670}}&lt;br /&gt;{{Source|handle=markshowa|date=2026-10-19|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1561808614618955938}} endsjeremy first doubted that 10 slots on one CPU was possible. {{Source|handle=endsjeremy|date=2026-10-19|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1561807910437265541}} ElectroHobo has the same server and confirms two PLX chips with 5 slots each; keep a group behind one switch, consecutive slots (1 to 5 on one switch). {{Source|handle=nathang6866|date=2026-10-19|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1561873069214273667}}&lt;br /&gt;{{Source|handle=nathang6866|date=2026-10-20|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1561967586978177035}}<br /> <br /> The PEX9797 management port is probably blocked or not wired (markshowa; his idea of an FPGA coordinator on it is untested). {{Status|Theory}} {{Source|handle=markshowa|date=2026-10-19|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1561879883930402920}} <br /> <br /> == Who runs one ==<br /> * Kaatchi runs cards in a SYS-4028GR in a garage: the fans are too loud even there and the server draws about 300 W with no load, much of it fans; he wants a fan tuning tool like the one for the Gigabyte G292. {{Source|handle=catchyuser|date=2026-10-09|link=https://discord.com/channels/1518634379604136178/1528513735575081070/1558163504878457046}}&lt;br /&gt;{{Source|handle=catchyuser|date=2026-10-15|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1560296770209062913}}<br /> * ElectroHobo mentions five 170HX in his Supermicro server. {{Source|handle=nathang6866|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1563727993896177735}}<br /> * endsjeremy expects these servers could run 170HX cards at full power. {{Source|handle=endsjeremy|date=2026-10-19|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1561809441026998323}}<br /> * A cheap source: markshowa&#039;s E5-2673 v4 cost 10 EUR. {{Source|handle=markshowa|date=2026-10-19|link=https://discord.com/channels/1518634379604136178/1534370835220004885/1561798498905755670}}<br /> <br /> == Related ==<br /> [[Platforms]], [[Risers and bifurcation#PCIe switch boards]], [[Gigabyte G292-Z20]], [[Cooling and power]].<br /> <br /> {{ImportFooter}}</div>

ImportBot