Standard benchmark matrix over 1, 2, 4 and 8 cards with llm-inference-bench, one model per card count; benchmark hygiene (temp 0 and speculative decoding inflate numbers)

Swimlane: Default swimlane
Column: Researching
Position: 13

Description

Reported by / on it: @djpandaman, @mountains1605, @gitanos, @shirogeek, @matstatman

Source: https://discord.com/channels/1518634379604136178/1528826330454429706/1558319513169170545

Wiki page: Benchmarks

Cards: both

Card D-067, seeded from the cmpunlocker Discord with credit (reference discord:1558319513169170545).

Sub-Tasks

Internal links

Comments