NVLink: Difference between revisions
More actions
NVLink on the CMP 170HX: fused off, not publicly unlocked, the missing board components, bridge part numbers and why it would only help diffusion. Discord Oct 2026.; sources: @lia_oc, @cute_sc2, @_orrorin, @ghostinthedatacenter_87281, @bigptoughneigh, @djpandaman, @wakeupmbrc1 |
NVLink on the CMP 170HX: fused off, not publicly unlocked, the missing board components, bridge part numbers and why it would only help diffusion. Discord Oct 2026.; sources: @lia_oc, @cute_sc2, @_orrorin, @onemillionkates, @_arcanum_, @ghostinthedatacenter_87281, @bigptoughneigh, @ob88594, @djpandaman, @wakeupmbrc1 |
||
| Line 17: | Line 17: | ||
* '''Fuse:''' <code>OPT_NVLINK_DISABLE</code> reads <code>0x2</code> on the 10 GB probe; CUTE measured the connector | * '''Fuse:''' <code>OPT_NVLINK_DISABLE</code> reads <code>0x2</code> on the 10 GB probe; CUTE measured the connector | ||
pins on 8 cards (all populated, pins connected to the core) and believes NVLink is blocked by GPU fuses. | pins on 8 cards (all populated, pins connected to the core) and believes NVLink is blocked by GPU fuses. | ||
{{Source|cute_sc2|2026-10-05|https://discord.com/channels/1518634379604136178/1534501623471145011/ | {{Source|handle=cute_sc2|date=2026-10-05|link=https://discord.com/channels/1518634379604136178/1534501623471145011/1556562554719965275}}<br /> | ||
{{Source|handle=cute_sc2|date=2026-10-05|link=https://discord.com/channels/1518634379604136178/1534501623471145011/1556562874342572094}} | |||
* '''Missing microcode:''' GA100 has three on-die MINION microcontrollers for link training; the 170HX VBIOS lacks the | * '''Missing microcode:''' GA100 has three on-die MINION microcontrollers for link training; the 170HX VBIOS lacks the | ||
bootstrap image. The VBIOS is not cryptographically signed, so injecting the image is the plan, but it is not done. | bootstrap image. The VBIOS is not cryptographically signed, so injecting the image is the plan, but it is not done. | ||
* '''Missing board parts:''' someone counted 5 resistors present on the A100 PCB that are absent on the 170HX, plus 3 | * '''Missing board parts:''' someone counted 5 resistors present on the A100 PCB that are absent on the 170HX, plus 3 | ||
terminations; the expected failure mode of unterminated NVLink is inability to train. | terminations; the expected failure mode of unterminated NVLink is inability to train. | ||
{{Source|_orrorin|2026-10-12|https://discord.com/channels/1518634379604136178/1518695328008638516/ | {{Source|handle=_orrorin|date=2026-10-12|link=https://discord.com/channels/1518634379604136178/1518695328008638516/1559264615911923833}} | ||
The 3.3 V NVLink clock generator parts are also missing; kater found U11 (the bridge clock generator, | The 3.3 V NVLink clock generator parts are also missing; kater found U11 (the bridge clock generator, | ||
GS7612S5MA-R, on the back of the PCB near the IO shield) is unpopulated, so the bridge clock never comes up. | GS7612S5MA-R, on the back of the PCB near the IO shield) is unpopulated, so the bridge clock never comes up. | ||
{{Source|ghostinthedatacenter_87281|2026-10-22|https://discord.com/channels/1518634379604136178/1518692084729511967/ | {{Source|handle=onemillionkates|date=2026-10-05|link=https://discord.com/channels/1518634379604136178/1534501623471145011/1556788730717675702}}<br /> | ||
{{Source|handle=_arcanum_|date=2026-10-07|link=https://discord.com/channels/1518634379604136178/1534501623471145011/1557248811351539713}}<br /> | |||
{{Source|handle=ghostinthedatacenter_87281|date=2026-10-22|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1562905010692558939}} | |||
== Status of the hardware mod == | == Status of the hardware mod == | ||
| Line 32: | Line 35: | ||
to share though." GhostInTheDatacenter has about 75% of the NVLink circuit components and is waiting on 50 milliohm | to share though." GhostInTheDatacenter has about 75% of the NVLink circuit components and is waiting on 50 milliohm | ||
resistors plus parts to re-enable the power stages; if it works he will document a BOM. | resistors plus parts to re-enable the power stages; if it works he will document a BOM. | ||
{{Source|bigptoughneigh|2026-10-23|https://discord.com/channels/1518634379604136178/1518695328008638516/1563312993414946827}} | {{Source|bigptoughneigh|2026-10-23|https://discord.com/channels/1518634379604136178/1518695328008638516/1563312993414946827}}<br /> | ||
{{Source|handle=ghostinthedatacenter_87281|date=2026-10-24|link=https://discord.com/channels/1518634379604136178/1518695328008638516/1563603290145423391}}<br /> | |||
{{Source|handle=ghostinthedatacenter_87281|date=2026-10-25|link=https://discord.com/channels/1518634379604136178/1518695328008638516/1563706962703683687}} | |||
OB88594's advice for the missing parts: add the components that are missing and follow the A100 layout. | |||
{{Source|handle=ob88594|date=2026-10-24|link=https://discord.com/channels/1518634379604136178/1518695328008638516/1563420061710950601}} | |||
mrpandagod is working on the software side after the RIR unlock and says it looks like it will work; he rewrote the SEC2 | mrpandagod is working on the software side after the RIR unlock and says it looks like it will work; he rewrote the SEC2 | ||
payload for NVLink during experiments. Nothing public as of 2026-10-28. | payload for NVLink during experiments. Nothing public as of 2026-10-28. | ||
| Line 40: | Line 47: | ||
A100-class PCIe cards use 3 bridge slots; each bridge carries 4 NVLinks (200 GB/s per bridge, 600 GB/s per pair), so an | A100-class PCIe cards use 3 bridge slots; each bridge carries 4 NVLinks (200 GB/s per bridge, 600 GB/s per pair), so an | ||
8-card rig would need 12 bridges. Compatible parts: PNY RTXA6000NVLINK-KIT (2-slot), PNY RTXA6000NVLINK3S-KIT (3-slot), | 8-card rig would need 12 bridges. Compatible parts: PNY RTXA6000NVLINK-KIT (2-slot), PNY RTXA6000NVLINK3S-KIT (3-slot), | ||
NVIDIA 900-53651-0000-000. {{Source|djpandaman|2026-10-06|https://discord.com/channels/1518634379604136178/1518692084729511967/ | NVIDIA 900-53651-0000-000. {{Source|handle=djpandaman|date=2026-10-06|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557070037209317407}}<br /> | ||
{{Source|handle=djpandaman|date=2026-10-06|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1557074851594899527}} | |||
Chinese clone bridges are on Alibaba for about $200; a Taobao supplier quoted RMB780 unbranded / RMB900 NVIDIA for the | Chinese clone bridges are on Alibaba for about $200; a Taobao supplier quoted RMB780 unbranded / RMB900 NVIDIA for the | ||
dual-slot 900-53651-0000-000, bought via Superbuy; on xianyu about $120 each. US retail is 150+ USD plus tax. | dual-slot 900-53651-0000-000, bought via Superbuy; on xianyu about $120 each. US retail is 150+ USD plus tax. | ||
| Line 49: | Line 57: | ||
Miséricorde: NVLink would help only diffusion workloads and would need higher power limits, so it would not change much | Miséricorde: NVLink would help only diffusion workloads and would need higher power limits, so it would not change much | ||
for LLM inference; for GLM 5.3 Flash in TP4 it might add roughly 20-25%. | for LLM inference; for GLM 5.3 Flash in TP4 it might add roughly 20-25%. | ||
{{Source|wakeupmbrc1|2026-10-27|https://discord.com/channels/1518634379604136178/1518692084729511967/ | {{Source|handle=wakeupmbrc1|date=2026-10-18|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1561400562036248677}}<br /> | ||
{{Source|handle=wakeupmbrc1|date=2026-10-27|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1564684409029005334}}<br /> | |||
{{Source|handle=wakeupmbrc1|date=2026-10-27|link=https://discord.com/channels/1518634379604136178/1518692084729511967/1564684449642193021}} | |||
Pairs only (TP2) on A100-class PCIe; combining more than two cards over bridges is unresolved. | Pairs only (TP2) on A100-class PCIe; combining more than two cards over bridges is unresolved. | ||
Latest revision as of 10:02, 30 October 2026
| Theory 8 GB and 10 GB | |
| Driver | n/a |
|---|---|
| Unlock build | none public |
| Steps | NVLink |
| Source | Discord message |
| Confirmations | None yet. |
NVLink is not publicly unlocked on the CMP 170HX, on either card. It is fused off and also missing board components. The 8 GB card has no NVLink path at all; the 10 GB card may be reachable with a hardware mod that is not public. Source: @lia_oc, 2026-10-22
What blocks it
- Fuse:
OPT_NVLINK_DISABLEreads0x2on the 10 GB probe; CUTE measured the connector
pins on 8 cards (all populated, pins connected to the core) and believes NVLink is blocked by GPU fuses. Source: @cute_sc2, 2026-10-05
Source: @cute_sc2, 2026-10-05
- Missing microcode: GA100 has three on-die MINION microcontrollers for link training; the 170HX VBIOS lacks the
bootstrap image. The VBIOS is not cryptographically signed, so injecting the image is the plan, but it is not done.
- Missing board parts: someone counted 5 resistors present on the A100 PCB that are absent on the 170HX, plus 3
terminations; the expected failure mode of unterminated NVLink is inability to train. Source: @_orrorin, 2026-10-12 The 3.3 V NVLink clock generator parts are also missing; kater found U11 (the bridge clock generator, GS7612S5MA-R, on the back of the PCB near the IO shield) is unpopulated, so the bridge clock never comes up. Source: @onemillionkates, 2026-10-05
Source: @_arcanum_, 2026-10-07
Source: @ghostinthedatacenter_87281, 2026-10-22
Status of the hardware mod
Big Ptoughneigh: "it works fine, but requires a very simple hardware mod (easier than the x16 mod). It's not my finding
to share though." GhostInTheDatacenter has about 75% of the NVLink circuit components and is waiting on 50 milliohm
resistors plus parts to re-enable the power stages; if it works he will document a BOM.
Source: @bigptoughneigh, 2026-10-23
Source: @ghostinthedatacenter_87281, 2026-10-24
Source: @ghostinthedatacenter_87281, 2026-10-25
OB88594's advice for the missing parts: add the components that are missing and follow the A100 layout.
Source: @ob88594, 2026-10-24
mrpandagod is working on the software side after the RIR unlock and says it looks like it will work; he rewrote the SEC2
payload for NVLink during experiments. Nothing public as of 2026-10-28.
Bridges (for when it works)
A100-class PCIe cards use 3 bridge slots; each bridge carries 4 NVLinks (200 GB/s per bridge, 600 GB/s per pair), so an
8-card rig would need 12 bridges. Compatible parts: PNY RTXA6000NVLINK-KIT (2-slot), PNY RTXA6000NVLINK3S-KIT (3-slot),
NVIDIA 900-53651-0000-000. Source: @djpandaman, 2026-10-06
Source: @djpandaman, 2026-10-06
Chinese clone bridges are on Alibaba for about $200; a Taobao supplier quoted RMB780 unbranded / RMB900 NVIDIA for the
dual-slot 900-53651-0000-000, bought via Superbuy; on xianyu about $120 each. US retail is 150+ USD plus tax.
Source: @djpandaman, 2026-10-07
Is it worth it?
Miséricorde: NVLink would help only diffusion workloads and would need higher power limits, so it would not change much
for LLM inference; for GLM 5.3 Flash in TP4 it might add roughly 20-25%.
Source: @wakeupmbrc1, 2026-10-18
Source: @wakeupmbrc1, 2026-10-27
Source: @wakeupmbrc1, 2026-10-27
Pairs only (TP2) on A100-class PCIe; combining more than two cards over bridges is unresolved.
Status
Theory: no public software or hardware unlock, only in-progress hardware experiments. The 8 GB card has no path.