How the 170HX compares

Members compared the unlocked CMP 170HX with what a buyer would otherwise consider: the RTX 3090, the A100, the DGX Spark, Apple's Mac Studio, and larger rigs built from V100, RTX 5060 Ti or RTX 5090 cards. Most figures are for the 8 GB card unlocked to 64 GB. Few runs used the same model, quant and engine on both sides, and llama.cpp or ollama results against the 3090 differ from vLLM results, so read every number with its engine and settings. The view most members share is under Positioning; prices and other options are on Alternatives at these prices and Price history and market.

Two matched runs are still wanted: one 170HX against an RTX 3090 on vLLM with matched quants (single stream, concurrency 4 and 16, prefill TFLOPS achieved), and one 170HX against a DGX Spark with NVFP4 prefill in vLLM and Qwen 3.6 or 3.8 27B decode on the same quant and engine.

Against the RTX 3090

RTX 3090: llama.cpp results trail it, vLLM results lead it

In a thread on 2026-08-03 about whether the card is slower than an RTX 3090:

• arnold civardagezen, who had no card, cited llama.cpp CUDA performance discussion #15013 (github.com/ggml-org/llama.cpp/discussions/15013), where a member's 170HX result trailed the 3090 in pp and tg. Scaling linearly from the 3090's 82 SMs he expected pp512 near 4,700 and tg near 280, and suspected the tensor cores were not being used. Source: @a.civardagezen, 2026-08-03
• giaki3003: prefill is in line with a nerfed A100 and inference is about 30% slower than an A100. Studebaker: prefill is compute-bound, decode bandwidth-bound. Source: @giaki3003, 2026-08-03
  Source: @_studebaker_, 2026-08-03
• WinterIsAmateur: llama.cpp does not use the BF16 tensor cores and the HBM well; with vLLM and W4A16 the 170HX is clearly faster, especially in prefill. Source: @winter_is_amateur, 2026-08-03
• DeeJay moved from 4x 3090 to 4x 170HX. The 170HX "handily beats the 3090 in nearly every test" and is about 2x faster in image and video generation, but it fails at llama.cpp, where its prompt processing is about 6x slower than in vLLM. Source: @deejay9983, 2026-08-03
• truwor measured about 2.5x a single 3090 on real (concurrent) workloads and wrote a comparison document (cmp170hx-vs-rtx3090.md); turbine and Samuel Hayden said vLLM works out of the box. Source: @truvv0r, 2026-08-03
  Source: @turbine, 2026-08-03
  Source: @samuel_hayden., 2026-08-03

Engine notes: llama.cpp on CMP cards. Image and video numbers: Image, video, voice and training.

Contested

Prefill against the 3090: compute-bound, and whose TFLOPS figure is right

• segmond, who uses llama.cpp, said prefill is compute-bound. He quoted the CMP 170HX at 4,480 cores at 1.15 GHz against 10,496 at 1.4 GHz for the 3090 (35.58 TFLOPS for the 3090 against 12.63 for the CMP), and his 3090 does prefill about 3x faster. Decode is memory-bandwidth-bound and there the 170HX beats the 3090, while with several cards the 3090 wins because of the PCIe bottleneck. Source: @segmond, 2026-08-07
• bayley_w disputed the compute figures: about 200 TFLOPS BF16 for the CMP 170HX against about 150 for the 3090. In pipeline parallel the micro-batches overlap, so prefill keeps the pipeline full. Source: @bayley_w, 2026-08-07

For measured BF16 throughput on one card, see the torch GEMM run under Positioning below (about 180 TFLOPS BF16) and Mmapeak.

Contested

ollama defaults: about 50 tok/s on the CMP against 55 on a 3090 (Qwen3.8 27B Q4_K_M)

• Naturidride quoted a comparison he had read: Qwen3.8 27B Q4_K_M on ollama defaults ran at about 50 tok/s on a 64 GB CMP 170HX and about 55 tok/s on a 24 GB RTX 3090, and the model loaded 2 to 3 times slower on the CMP. Source: @darcidride, 2026-08-19
• DaLZy replied that the CMP is faster and that people judge it from llama.cpp and ollama results. Source: @dalzyyy, 2026-08-19
• Morrowmake expected higher single-card decode from the HBM bandwidth, and replacedriver47 said a well-tuned CMP should win on decode but the 3090 beats it handily on prefill. Source: @voidgiraffe, 2026-08-19
  Source: @replacedriver47, 2026-08-19

clee linked a related post in #general. Source: @c133, 2026-08-19

Contested 8 GB

Qwen3.8 27B Q4 at c=1 without speculative decoding: the CMP about 15% ahead of a 3090

cressida ran a single-user (c=1) test of Qwen3.8 27B Q4 on both cards without speculative decoding: the CMP 170HX, unlocked to 64 GB (2.7x the 3090's 24 GB), beat an RTX 3090 by about 15%. The exact tok/s figures are only in the attached screenshots. Miséricorde called that very low for c=1 on a CMP and posted a screenshot of 100+ tok/s with a DFlash draft model; raw comparisons without MTP or DFlash are valid, he said, but should say so, and he asked which HBM frequency cressida ran. cressida expects the CMP to gain more under concurrency, and Miséricorde agreed that the larger memory helps the KV cache for concurrent sessions. Drafting methods: Speculative decoding on the 170HX.

Reported 8 GB
Source: @cressida1881, 2026-09-19
Source: @wakeupmbrc1, 2026-09-19

"A 3090 with more VRAM": similar compute, much more memory bandwidth, fewer features

• 2026-07-26: asked about a 48 GB Quadro RTX 8000 against an unlocked 8 GB card (64 GB), diabeetee would pick the CMP, especially for a single-card setup. ian j'j'j replaced two RTX 3090s with one 170HX 8 GB card: two 3090s did not have enough VRAM, 64 GB fit his needs better, and close to $1000 was left over. Source: @diabeetee, 2026-07-26
  Source: @mrmannyatx, 2026-07-26
  Source: @wxy3657, 2026-07-26
• 2026-07-30: asked whether the 170HX is a 3090 with more VRAM, Luna said "pretty much". makesier (weicj) agreed for compute but pointed to the much faster HBM2e bandwidth, and said that even with prices rising it is the cheapest 64 GB of HBM on a single card. Arcanum described it as a nerfed A100. Source: @rogersmith2026, 2026-07-30
  Source: @lunaroseoc, 2026-07-30
  Source: @makesier, 2026-07-30
  Source: @_arcanum_, 2026-07-30
• 2026-08-09: mfj4 asked what a 3090 can do that a 170HX cannot. bayley_w: the 3090 works on Windows and has NVENC, RT cores and raster engines; the 170HX has no fans of its own, does not really idle while 3090s do, and is limited on PCIe where the 3090 has 4.0 x16. His rule of thumb is to think of the 170HX as a 3090 with extra VRAM; on 2026-08-19 he called it slightly more than a 3090 but not directly comparable, because it has no raster engine. On price, robbbbbb7973 put the 170HX at about 2x a 3090 and a gamble on whether it unlocks properly, mfj4 saw them near 1:1, and Aeiou said that at 2x the price you get slightly more than double the memory with roughly similar compute. ju5t5omedude added gaming as a reason to keep a 3090. Source: @mfj4, 2026-08-09
  Source: @bayley_w, 2026-08-09
  Source: @robbbbbb7973, 2026-08-09
  Source: @91d5a66bffc978331fac75b68fa86aba, 2026-08-09
  Source: @ju5t5omedude, 2026-08-09
• 2026-08-19: giaki3003 held that the 3090 has more compute and that the 170HX's advantage is bandwidth. Source: @giaki3003, 2026-08-19 DaLZy countered that the 170HX has far more tensor throughput: 808 INT4 TOPS for the 170HX (with 4,480 cores) against 570 for the 3090. GA100 and GA102 SMs differ so much, including in how CUDA cores are counted, that spec comparisons mislead, he said, and old CUDA benchmarks used broken GA100 kernels while "the 170tune one is accurate" (170tune). Source: @dalzyyy, 2026-08-19 Vcore noted that which card wins depends on whether the software uses the tensor cores, and wondered (untested) whether the disabled CUDA cores sit unused on the die. Source: @vcore2191, 2026-08-19
  Source: @letterc67, 2026-08-19
• 2026-09-21: xtj7 gets nearly the same LLM performance from one 170HX as from 2x RTX 3090 at less than half the power draw; turbine explained that the 3090 is GA102, whose smaller tensor cores need higher clocks for the same throughput. Source: @xtj7, 2026-09-21
  Source: @turbine, 2026-09-21

Contested 8 GB and 10 GB

Against the A100

About 30% slower than an A100; mixed against 2x 3090

giaki3003 said on 2026-08-11 that the 170HX is about 30% slower than an A100: take A100 numbers and remove about 30% for the SM count. clee pointed to mfj4's comparison of one CMP 170HX against 2x RTX 3090 on qwen3.6-27B-FP8 (on Qwen3.6 27B and 35B-A3B on one card (July and August 2026)), and mfj4 said the same test with gemma4 31B flipped the result, with the 3090s slightly ahead. DmitryKuzakov asked for vLLM --max-num-seqs 16 TTFT numbers; no TTFT figures followed, and the only reply was a GLM 5.2 concurrency figure (200 tok/s at concurrency 16).

Reported
Source: @giaki3003, 2026-08-11
Source: @c133, 2026-08-11
Source: @mfj4, 2026-08-11
Source: @dmitrykuzakov3756, 2026-08-11

"A80 64GB": A100 bandwidth with about 75% of the compute, 8-card servers built

On 2026-07-24 LetterC67 described the unlocked card as A100 memory bandwidth with about 75% of the compute and put its real value at $1k or more. DeeJay compared it with an RTX 5000 Pro Blackwell without INT4 tensor cores and with slower PCIe, and judged $1500 fair against an 8k 72 GB card. On 2026-07-25 giaki3003 said 8x 170HX servers had already been built and run in parallel at about 30% below an A100, and quark2world dubbed the unlocked card an "A80 64GB" for its reduced SM count. Rigs of that size: Large rigs (8 or more cards).

Reported
Source: @letterc67, 2026-07-24
Source: @deejay9983, 2026-07-24
Source: @quark2world, 2026-07-25
Source: @giaki3003, 2026-07-25

About 60 to 70% of an A100

ProfDD (2026-07-24) said a 170HX running LLMs performs at about 70% of an A100, with the limit of PCIe 1.0; Rickcal separately posted "12.5 TFLOPS FP32 and 6.2 TFLOPS FP64" without stating the setup or whether the card was unlocked. quark2world (2026-08-07) measured about 66% of an A100 in nearly everything he tested, including large-context agentic LLM inference, with about 33% of the compute cores disabled. lrdimm (2026-08-19) also put it at about 66% of an A100 "right now", comparable to Ampere cards but with HBM2e. On 2026-10-02 Busulle asked whether a 74-SM unlock makes the card about 68% of an A100 (74 of 108 SMs); CUTE agreed and said it is a little more, because the CMP runs higher clocks and more power per SM, while clee had no real A100 to compare against. These are estimates without a shared method. The 74-SM unlock: Full SM unlock.

Reported 8 GB and 10 GB
Source: @profdd, 2026-07-24
Source: @rrickk420, 2026-07-24
Source: @cute_sc2, 2026-08-07
Source: @jimtjames, 2026-08-07
Source: @dalzyyy, 2026-08-07
Source: @quark2world, 2026-08-07
Source: @petrikrohn_26497, 2026-08-07
Source: @overrflow, 2026-08-07
Source: @lrdimm, 2026-08-19
Source: @bayley_w, 2026-08-19
Source: @sh.ad, 2026-08-19
Source: @busulle, 2026-10-02
Source: @cute_sc2, 2026-10-02
Source: @c133, 2026-10-02

"1.5 TB/s Ampere": about 60% of a GA100, and power-limited

On 2026-08-22 segmond noted that popular articles were calling the card an A100; DeeJay countered that it is about 60% of a GA100 with some TFLOPS still missing, sold for $1500 ("thats the part they leave out"). A TeksEdge post on X (x.com/TeksEdge/status/2090960336275083509) was cited as one driver of the price rise. On 2026-08-23 ST, in a thread comparing Blackwell with Ampere, pointed out that the 170HX is not ordinary Ampere but "1.5tb/s ampere": its HBM2e bandwidth of about 1.5 TB/s is its strength for LLM decode despite the older architecture. On 2026-08-28 quark2world said the CMP cards are slow in many workloads, where a 4090 or 5090 beats them by far, and that they are heavily limited by power draw even with a mining BIOS at the full power limit; at around $2k he thought a 4090 the better buy for many users. cressida summed the card up as roughly 150 to 200 W for 64 GB of VRAM. Power limits: Undervolting and power limits.

Reported 8 GB and 10 GB
Source: @segmond, 2026-08-22
Source: @deejay9983, 2026-08-22
Source: @matstatman, 2026-08-22
Source: @artistosu, 2026-08-22
Source: @stcord, 2026-08-23
Source: @quark2world, 2026-08-28
Source: @cressida1881, 2026-08-28

Against the DGX Spark and the Mac Studio

DGX Spark: about 6x less memory bandwidth at similar FP8 throughput

Takka's framing of value: the DGX Spark launched at USD 5k with about 6x lower memory bandwidth (so a 6x lower decode ceiling) and roughly the same FP8 throughput (238 TFLOPS for the Spark against 205 for the 170HX 8 GB card); he does not count FP4 as real yet. domrockt2 owned a Spark, found it "not far away from this hack job", and sold it to fund 170HX cards. Josh's counterpoint: the Spark is a supported retail product and the CMP is a hack on binned memory.

Reported 8 GB
Source: @spectakkular, 2026-10-05
Source: @domrockt2, 2026-10-05
Source: @_____josh_____, 2026-10-05
Source: @voidgiraffe, 2026-10-05

Four CMPs against two DGX Sparks in August: members agreed the CMPs are cheaper and faster

In late August members agreed that 4 CMPs are cheaper and faster than 2 DGX Sparks. SpiralGlide reported 4x DGX Spark running GLM int4/int8 at about 30 tok/s single stream and 60 aggregate, with only about 200k of KV cache at fp8. Dino (2026-08-23) said dual MI50s rival the 170HX in price and token generation but need good PCIe lanes for tensor splitting plus ROCm work (he got two 32 GB MI50s for about $200 as a fluke); Bart saw $500 and up on eBay and $400 EXW on Alibaba, and nearly bought an 8x V100 32 GB NVLink server for $5k before the 170HX news. machinezero linked a Reddit post of DeepSeek V4 Flash on an M2 Ultra; others noted the Macs' weak prefill.

Reported 8 GB
Source: @voidgiraffe, 2026-08-22
Source: @jo29_29, 2026-08-22
Source: @artistosu, 2026-08-22
Source: @machinezer0, 2026-08-23
Source: @dino_38975, 2026-08-23
Source: @justbendev, 2026-08-23
Source: @meromgreen, 2026-08-23
Source: @spiralglide, 2026-08-24

Prefill against a DGX Spark running NVFP4

• Civiloid: the CMP 170HX is about 0.65 of an A100 PCIe, roughly 132 TFLOPS FP16/BF16 against 118.8 for a DGX Spark, and the Spark doubles at FP8 and doubles again at NVFP4. On the CMP there is no gain from BF16 to FP8; only INT8 (2x) and INT4 (another 2x) speed up. In his tests the CMP is about 15% slower in prefill than a Spark running NVFP4 on vLLM, while Spark decode is about 1/6 of the CMP's; this would change if about 100 SMs could be unlocked. Source: @civiloid, 2026-08-24
• giaki3003 disputed the prefill claim, and Petar said he would not go below Q8/FP8. Source: @giaki3003, 2026-08-24
  Source: @spale777, 2026-08-24

Formats and quants: Quantization on the 170HX.

Contested

Mac Studio M5 Ultra (256 GB, $9,499) against 4x CMP 170HX

After the M5 Ultra launch on 2026-08-25 (256 GB at $9,499, 512 GB expected in October):

• d3su called it a better deal than 4x CMP at current prices, and segmond considered selling his cards. Source: @.d3su, 2026-08-25
  Source: @segmond, 2026-08-25
• Petar: prefill is compute-bound. An M3 Ultra does 300 to 500 tok/s PP on DeepSeek V4 Flash, a DS4F braindead quant on the Mac about 500 tok/s PP, and in agentic workloads input tokens are about 20x output, so PP and caching dominate. He expected Mac prefill at or below a DGX Spark, with no native FP4 or FP8. Source: @spale777, 2026-08-25
• giaki3003 measured prefill on a base M4 at 300 tok/s (50 on large prompts) and on a base M5 at 900 (400 on large prompts), projected 3.6k to 6k PP for the M5 Ultra (two M5 Max: 36 cores, 80 neural accelerators), and said CMP prefill would be worse regardless. Source: @giaki3003, 2026-08-25
• DeeJay preferred a $10.7k 256 GB Mac over 4 CMPs at $8000 for its other uses. Source: @deejay9983, 2026-08-26
• bayley_w ran GLM-5.2 on a 512 GB Mac at about 30 tok/s prefill and put the M5 Ultra at only about 140 TFLOPS. Source: @bayley_w, 2026-08-26

Petar's and giaki3003's conclusion was that 4x CMP 170HX is still the best gamble. ST, machinezero and jac also took part. Source: @stcord, 2026-08-25
Source: @machinezer0, 2026-08-25
Source: @jecquas, 2026-08-25

Contested

8x 170HX against a 512 GB Mac Studio or a GB300 workstation

Impactic (2026-08-30) considered selling his 170HX cards for a 512 GB Mac Studio; Studebaker estimated that Mac at $20k to $25k and said 8x 170HX give much more compute. Rickcal suggested that 4 CMPs on a PLX switch with Gen5 RAID 0 storage could compete with MSI's GB300 workstation (748 GB, about $100k: 252 GB HBM3e plus 496 GB LPDDR5X).

Theory
Source: @impactic, 2026-08-30
Source: @_studebaker_, 2026-08-30
Source: @rrickk420, 2026-08-31

Two DGX Sparks against four CMPs at 150 W (arca.live)

gpulover shared a post from the Korean AI Homelab board (arca.live/b/aihomelab/181906661) comparing 2x DGX Spark against 4x CMP 170HX on DeepSeek V4 (vision Exp), with the CMP cards capped at 150 W each. No numbers were posted in the channel; see the linked thread.

Reported
Source: @artistosu, 2026-09-03

Prefill: Apple M5 Max against the CMP

xtj7 (2026-09-17): an M5 Max gets about 500 tok/s prefill on Qwen 3.8 27B at Q4, and an M5 Ultra would at most double that to about 1k, still below a dual 3090 setup and below a CMP setup with a single GPU. Macs have great memory speed for decode but very slow prefill. Against the DGX Spark, memory bandwidth is where the CMPs excel (about 270 GB/s on the Spark, per Takka).

Reported
Source: @xtj7, 2026-09-17
Source: @spectakkular, 2026-09-17

Value in September and October: a Mac Studio, two DGX Sparks or four CMPs

cibernox (2026-09-23): the new M5 chips have about 4x the prior prefill, but a 256 GB / 4 TB Mac Studio costs about 14.5k EUR, and the DGX Spark's 270 GB/s bandwidth is its weak point. _L_ noted that almost 3 DGX Sparks fit that budget, but an unlocked CMP 170HX is still much better value if you accept a PC build, and 4x CMP fit a rig budget even at inflated prices. Takka estimated two Sparks at 2 to 4x slower decode than the Mac (1.2 TB/s against 540 GB/s aggregate, before communication losses). On 2026-10-02 and 10-03: Landon owns 2 DGX Sparks and is considering 4x 170HX; domrockt2 sold his DGX Spark for a 4-card 170HX setup and does not look back; Ikyo said that so far a Mac Studio appears able to hold its own against a single 170HX (no model or numbers); Tokukarin said CMPs are much faster than a Mac and were much cheaper when he bought, though a Mac may suit people who are not good with hardware; orrorin and SeriousRising said CMPs cost less out of pocket if you already have the supporting hardware, but take time.

Reported 8 GB
Source: @cibernox, 2026-09-23
Source: @lia_oc, 2026-09-23
Source: @spectakkular, 2026-09-23
Source: @domrockt2, 2026-10-02
Source: @landontg, 2026-10-02
Source: @ikyo, 2026-10-03
Source: @_orrorin, 2026-10-03
Source: @tokukarin, 2026-10-03
Source: @seriousrising, 2026-10-03

Qwen 3.8 27B: 170HX against a Mac Studio (screenshots only)

Ikyo posted two screenshots on 2026-10-03 comparing Qwen 3.8 27B served on a 170HX and on a Mac Studio. The numbers are only in the images and were not transcribed in the chat; Busulle took part in the thread.

Reported
Source: @ikyo, 2026-10-03
Source: @busulle, 2026-10-03

Against other hardware

Performance class estimates in July: A40-class, about 2/3 of an A100's SMs

quark2world said the benchmarks he had seen by 2026-07-20 put the unlocked card in the A40 speed class; SkirtChaser called it effectively a nerfed A100 64GB whose 64 GB seems stable and usable. Luna put it at about 2/3 the CUDA cores and SMs of an A100 40GB with nearly 50% more VRAM, and arnold civardagezen said about 20% of the CUDA cores are dead. SkirtChaser estimated that an MI50 32GB gives about 25 to 35% of the 170HX's prefill and about 60% of its generation speed (from numbers posted for the unlocked card). Luna: three 3090s are faster but draw about 1,400 W against about 240 W and need 48 PCIe lanes against 16.

Theory 8 GB
Source: @quark2world, 2026-07-20
Source: @skirtchaser, 2026-07-20
Source: @lunaroseoc, 2026-07-20
Source: @a.civardagezen, 2026-07-20

Qwen 3.6 27B decode: 170HX against DGX Spark, 2x MI50 and 2x 5060 Ti

• roger_dammit, who owns a DGX Spark, said it is poor for Qwen 3.6 27B dense and that 170HX benchmarks put the 170HX at about 2 to 4x the Spark's decode rate. He noted the Spark has about a quarter of the memory bandwidth, and linked github.com/Highwayaiexpose/CMP-170hx-64gb-LLM-benchmarks and a r/LocalLLaMA post on an $1,800 P2P build. Source: @roger_dammit, 2026-07-23
• arnold civardagezen and chris.via.egnatia said it is closer to 2x. arnold civardagezen estimated Qwen 27B Q4_K_M on one 170HX at about 50 to 60 tok/s, similar to Qwen 27B NVFP4 at TP=2 on two 5060 Ti cards (methodology differs). Source: @a.civardagezen, 2026-07-23
  Source: @chris.via.egnatia, 2026-07-23
• CUTE reported about 50 tok/s for Qwen 3.6 27B dense on 2x MI50. Source: @cute_sc2, 2026-07-23

Single-card Qwen 3.6 numbers from that time: Qwen3.6 27B and 35B-A3B on one card (July and August 2026).

Contested 8 GB

Reference rig: GLM 5.2 on 16x V100 32 GB (two DGX-1)

bayley_w ran GLM 5.2 on two DGX-1 nodes (8x V100 SXM2 32 GB each) at TP=4 PP=4, AWQ with INT4 experts, FP8 attention and shared experts, and FP8 KV cache: 3,000 tok/s prefill and 40 tok/s decode at 262k context, holding a flat 40 tok/s to the full quarter-million context. Profiling showed that going much past 50 tok/s is hard on a model this deep, because norms and allreduce in each of the 79 layers dominate. Groups of 4 NVLinked cards on Chinese SXM2 boards with PCIe between the groups would work, since the TP groups are 4 cards. The cost was about $10K; DeeJay noted 16 cards draw about 4,000 W and judged the 170HX the better deal if stable.

Reported
Source: @bayley_w, 2026-07-24
Source: @deejay9983, 2026-07-24

Reference rig: CPU-offload GLM on a Threadripper Pro with DDR4 and 4x RTX 3090

DeeJay ran GLM 4.8 on a Threadripper Pro with 256 GB DDR4 and 4x RTX 3090, keeping 80% of the weights in system RAM and 20% plus prefill and KV cache on the GPUs, with MTP: about 13 tok/s decode. The 4x 3090 reached about 3,000 tok/s prefill on vLLM. A dual Xeon 6 MRDIMM build (about 1.5 TB/s, roughly $60k) was estimated at 40 to 45 tok/s decode in FP8 but never tested; Intel's SGLang path supports only FP8 or INT4, not llama.cpp Q4/Q6. Eksitus took part in the thread.

Reported
Source: @deejay9983, 2026-07-24
Source: @eksitus, 2026-07-24

Reference rig: 16x RTX 5060 Ti 16 GB at TP8 x PP2

machinezero posted a non-CMP reference rig of 16 RTX 5060 Ti 16 GB cards at tensor parallel 8 and pipeline parallel 2 (r/LocalLLaMA): 500k context, about 4,000 tok/s prefill up to 500k context, and 100 to 150 tok/s decode, averaging 140 in a DeepSeek harness. He said the setup is good if each card idles under 10 W. Arcanum asked how to feed Gen5 x8 to all 16 cards, and Schaka said Gen4 is easy with boards around 500 EUR that give each card x16 and the host x16, four of them.

Reported
Source: @machinezer0, 2026-08-20
Source: @_arcanum_, 2026-08-20
Source: @schakaqt, 2026-08-20

RTX 5090 on Qwen3.8-27B: similar decode, prefill 3 to 4x faster than the CMP

Miséricorde (2026-08-30) got slightly faster decode on 27B on the CMP than on a 5090 running Quasar/ninfer, but prefill 3 to 4x slower; he got 6K to 8K tok/s prefill on the 5090. DUFRelic: W8A8 has better prefill than weight-only quants. Phoenix17: INT8 keeps the prefill speedup and the parallelism needed for coding.

Reported
Source: @wakeupmbrc1, 2026-08-30
Source: @phoenixone7, 2026-08-30
Source: @dufrelic, 2026-08-30

4x CMP at 200 tok/s against a tinygrad tinybox

Against a tinygrad tinybox announcement (x.com/__tinygrad__/status/2103368229678891029), ST linked Morrowmake's post of 200 tok/s on 4 CMPs (x.com/Morrowmake/status/2103388350392713278; the model is not named in that thread). Lyubo thought prefill was 400; Takka said an update in another channel shows higher numbers, and Lyubo then called 2k prefill very good. cibernox says 4x CMP 170HX bought under $2,000 each should match a tinybox at about a quarter of the price, more with PCIe Gen4 and comfortable TP=4.

Reported
Source: @stcord, 2026-09-25
Source: @voidgiraffe, 2026-09-25
Source: @spectakkular, 2026-09-25
Source: @lubakmanqk, 2026-09-25
Source: @cibernox, 2026-09-25

Positioning

What to expect: a 3080-class card with a lot of VRAM, slower than a 3090 for models under 24 GB

After the July 2026 Qwen3.6-27B benchmarks, several members agreed that one unlocked 170HX is slower than an RTX 3090 for models that fit in 24 GB, and is not A100 performance. Its value is for workloads that need more than 24 GB and up to 64 GB on one card: bigger LLM quants and 40 to 60 GB image or video diffusion. Samuel Hayden said about 35 tok/s on Qwen3.6-27B matches what he gets on 2x A4500, and chris.via.egnatia said this confirms another member's results with a 35B model. giaki3003 summed it up as "like a 3080 with a lot of VRAM (and a lot of random other issues)". Eksitus: "we just really need to know what we are buying: more vram, but not A100 performance."

Proven 8 GB and 10 GB
Source: @eksitus, 2026-07-22
Source: @samuel_hayden., 2026-07-22
Source: @chris.via.egnatia, 2026-07-22
Source: @giaki3003, 2026-07-22

Torch GEMM on an unlocked 8 GB card: about 180 TFLOPS BF16

gym (2026-07-23) ran the zzc0721/torch-performance-test-data GEMM test on an unlocked 170HX 8 GB card (64 GB). Peak results: FP32 12.72 TFLOPS (8192), TF32 81.35 (8192), FP16 175.79 (4096), BF16 180.09 (4096); the 1024-size results were lower (FP32 10.42, TF32 54.47, FP16 87.02, BF16 110.18). He rated it around an RTX 4090 in TF32 and BF16, about 20% slower than a 5090, much slower in plain FP32 (CUDA cores), and about 75% of an A100 40GB PCIe rather than the expected 65%. This test reads from HBM, so it is more representative than Mmapeak, which only issues mma instructions with operands in shared memory.

Reported 8 GB
Source: @jimtjames, 2026-07-23

"Basically 64 GB A100s": positioning summaries from August

On 2026-08-04 and 08-05, formatme said many had expected an 80 GB unlock of the 10 GB card that had not happened, which hurt the price (The 10 GB card: provenance and the 80 GB question). arnold civardagezen said NVLink does not work and that the "fuses" are read-only values whose physical state is unknown, with people looking into Gen3 and NVLink. CUTE said that if PCIe and BAR are fixed he would replace his MI50s, and machinezero said that having no Gen4 x16 or NVLink out of the box limits the market. On 2026-08-29 lrdimm said that for inference the unlocked cards are basically 64 GB A100s, ranking non-datacenter GPUs below CMPs below the A100, with the RTX Pro 6000 still ahead; RedPandaMining said they have higher memory bandwidth than an RTX Pro 6000; segmond called them still limited but the best value. Rickcal said no other card does inference under 200 W at maximum load (an MI250X needs a 48 V rail).

Reported 8 GB and 10 GB
Source: @cute_sc2, 2026-08-04
Source: @a.civardagezen, 2026-08-04
Source: @machinezer0, 2026-08-05
Source: @counterstrikeaimgod, 2026-08-05
Source: @lrdimm, 2026-08-29
Source: @segmond, 2026-08-29
Source: @redpandamining, 2026-08-29
Source: @rrickk420, 2026-08-30

Claim: only three GPUs beat the unlocked 64 GB card

• Petri Krohn (2026-08-09): only three GPUs beat the unlocked 64 GB 170HX in all respects for local LLMs, all over $10,000: A100 PCIe 80GB (about $10,000), RTX Pro 6000 96GB (about $12,000) and H100 PCIe (about $30,000). He cited the Highwayaiexpose/CMP-170hx-64gb-LLM-benchmarks repository as showing the 170HX beating the RTX Pro 6000 and RTX 5090. Source: @petrikrohn_26497, 2026-08-09
• bayley_w countered that it looks great until you need tensor parallel or mainline vLLM support, and that the benchmark has no prefill numbers and no token generation at 128k. Source: @bayley_w, 2026-08-09

PJ asked whether tensor parallelism will work on multi-card setups; it was not answered in that thread. Source: @zingy90210, 2026-08-10 Multi-card layouts: Parallelism over the PCIe link.

Contested 8 GB

TOPS figures and multi-card prefill overlap

DeeJay (2026-09-01) put the M5 Ultra's neural figure at 386 TOPS. bayley_w asked which number format that figure was in, said the 170HX has a mode rated at 1600 TOPS, and argued that during prefill all cards work together because the micro-batches overlap, so 4x 170HX deliver much more than the conservative 400 TOPS INT8 number. These are rated figures, not measurements on the 170HX.

Reported
Source: @deejay9983, 2026-09-01
Source: @bayley_w, 2026-09-01

Does GA100 have INT4 tensor cores?

• DeeJay (2026-08-27): the cards have no INT4 tensor cores but do have INT8, so INT8 weights may be as fast as INT4. Source: @deejay9983, 2026-08-27
• Petri Krohn: GA100 has INT4 tensor cores but no NVFP4, citing the A100 whitepaper (page 15): INT8 624/1,248 TOPS, INT4 1,248/2,496 TOPS (dense / 2:4 sparse). He believes FP64, FP32, BF16, FP16, INT8 and INT4 share the tensor cores at double throughput per halving of bits, with separate CUDA cores. Source: @petrikrohn_26497, 2026-08-28
• Phoenix17: INT4 runs packed on the same 8-bit tensor cores. Source: @phoenixone7, 2026-08-28

These are A100 specifications, not measurements on the 170HX. More on data types: Compute, data types and missing features.

Contested

Idea: Fast Walsh-Hadamard transform plus INT4 tensor cores

Petri Krohn (2026-09-22) argued that inference will move to Fast Walsh-Hadamard transforming both weights and activations and doing the math on INT4 tensor cores with 2 or 3-bit weights (W2A4 or W3A4), citing about 800 TOPS INT4 on the 170HX (not measured). He proposed adding a generic FWHT to llama.cpp to rotate and quantize activations on the fly, and said the optimal 2-bit encoding is +3/+1/-1/-3 rather than Bonsai 2's ternary values, because zero loses its special meaning once weights are Hadamard-transformed. Untested.

Theory
Source: @petrikrohn_26497, 2026-09-22

What limits the card: compute, bandwidth or PCIe P2P latency

• segmond (2026-08-08): prompt processing is compute-bound and token generation is bandwidth-bound; latency is mostly irrelevant, which is why GPU specs do not list it. Source: @segmond, 2026-08-08
• Jeremy (Ends): latency, especially P2P over PCIe across several cards, is the main limit for MoE use. Source: @endsjeremy, 2026-08-08

Background: P2P.

Contested

View: the SM80 architecture, not PCIe or NVLink, is the main limit

On 2026-08-13 DeeJay said the biggest challenge from here is software, since new models are designed around Hopper and Blackwell and Ampere is being left behind (workarounds exist for now); Con3po expected AWQ quants to stay supported for a few more years; Humvee55: "we are gonna end up making our own quants", and DeeJay said he already does. On 2026-09-01 insomniac argued that PCIe and NVLink are not nearly the bottlenecks people think on this card, and that the bigger concern is the ageing SM80 (Ampere GA100) architecture. Opinion, no measurements.

Theory
Source: @deejay9983, 2026-08-13
Source: @con3po7016, 2026-08-13
Source: @scallion_9883, 2026-08-13
Source: @ins_mniac, 2026-09-01

Gen2 x16 against Gen4 x16: bandwidth and value

• insomniac (2026-09-01): the current unlock gives at most about 8 GB/s per direction at Gen2 x16, against about 31.5 GB/s at Gen4 x16, so multi-card setups are weaker than DGX Spark clusters (200 Gb/s networking). Source: @ins_mniac, 2026-09-01
• Schaka countered that Sparks have much lower memory bandwidth. Source: @schakaqt, 2026-09-01
• Con3po posted a per-$1000 table (A100 80GB at $16k against the 170HX at $1.1k): 57.7 GiB usable HBM and 1454 GB/s per 170HX, P2P only 1.53 GB/s per card against about 600 GB/s NVLink P2P on the A100; 8 cards at $8,800 roughly match the HBM capacity of about $103k of A100s. Source: @con3po7016, 2026-09-01
• Miséricorde: for 1 to 2 card local setups the CMP is the best price, performance and HBM per dollar even at $2000 and up, but it scales poorly. Source: @wakeupmbrc1, 2026-09-01
• nightdays: at PCIe 1.1 x4 vLLM is already enough, and DeepSeek V4 Flash 0731 runs on 3 cards at 350 tok/s with 8 concurrent requests. Source: @.nightdays, 2026-09-02

Link speeds: Gen2 link retrain and P2P.

Contested

How many cards make sense

• Petri Krohn (2026-08-27): there is little point owning more than one 64 GB 170HX for anything other than Qwen 3.8 27B. For home labs the RAM to hold MoE experts dominates the cost (64 GB DDR5 about $1000, near the price of a 64 GB card), the new flash MoEs (GLM 5.3 Flash, Qwen 3.8 Flash Next, DeepSeek V4 Flash) score little above 27B, and 4 to 8 cards mean 1000 to 2000 W and noise. Source: @petrikrohn_26497, 2026-08-27
• Waka324 cited usable context over 500K and large codebases; segmond strongly disagreed with Petri (he runs DeepSeek V4 Flash at full quant and says it crushes 27B); Studebaker called 27B a great "worker bee" and stressed efficiency per watt. Source: @cmdrwaka324, 2026-08-27
  Source: @segmond, 2026-08-27
  Source: @_studebaker_, 2026-08-27
• giaki3003 noted FreeToken runs MoEs from DDR4 easily. Source: @giaki3003, 2026-08-28
• On 2026-09-04 and 09-05: JJ compared the cost per GB favourably with a Mac mini; insomniac doubted 27B's agentic usefulness; Morrowmake said the optimal use is a single card for Qwen 3.8 27B, and above 4 to 8 cards alternatives win; Phoenix17 sees sweet spots at 2 and 4 cards (4 run GLM 5.3 and the DeepSeek flash models, 2 run 6 parallel Qwen 3.8 27B agents or ComfyUI plus Qwen); prefix10k found 3 cards for GLM-5.3-Flash not worth it against Qwen 3.8 on one. Source: @jj48., 2026-09-04
  Source: @ins_mniac, 2026-09-04
  Source: @voidgiraffe, 2026-09-04
  Source: @phoenixone7, 2026-09-05
  Source: @prefix10k, 2026-09-05

Rigs past four cards: Large rigs (8 or more cards).

Contested

FP64 OpenCL as a reason to buy the 8 GB card

Luna (2026-09-06) said she does not need much VRAM and would be happy with the 8 GB card for its "over 6 TFLOPS" of FP64 OpenCL compute, a figure she did not measure. Phoenix17 suggested a CPU route (a 9950X does about 3 TFLOPS FP64, EPYC about 614 GB/s). Luna replied that desktop CPUs top out near 128 GB/s and that filling EPYC memory channels is expensive (about $800 per 32 GB module).

Theory 8 GB
Source: @lunaroseoc, 2026-09-06
Source: @phoenixone7, 2026-09-06

Related

CMP 170HX, 8 GB card, 10 GB card, Variant comparison, Alternatives at these prices, Price history and market, Buying, Compute, data types and missing features, Benchmarks, Mmapeak, Quantization on the 170HX, llama.cpp on CMP cards, Speculative decoding on the 170HX, Parallelism over the PCIe link, Large rigs (8 or more cards), Image, video, voice and training, Qwen3.6 27B and 35B-A3B on one card (July and August 2026), Qwen3.8 27B on 1x 8 GB (W4A16 vLLM), Undervolting and power limits, P2P, NVLink, Other CMP cards and other NVIDIA GPUs.

Imported from the cmpunlocker Discord with credit; see Credits.