Llm-visuals: Difference between revisions
More actions
Import: llm-visuals, @trustinter; repos DingoOz/llm-visuals |
Import: llm-visuals, @trustinter; repos DingoOz/llm-visuals |
||
| Line 1: | Line 1: | ||
{{DISPLAYTITLE:llm-visuals}} | |||
{{Status|Reported}} {{Cards|both}} | {{Status|Reported}} {{Cards|both}} | ||
Latest revision as of 08:38, 30 October 2026
Reported 8 GB and 10 GB
llm-visuals is a live terminal dashboard for a local inference server. It auto-detects llama-server, ollama, vLLM, SGLang and several other engines and shows decode and prefill speed, time to first token, per-GPU utilisation, power, HBM temperature, clock-throttle reasons and the VRAM split between weights and KV cache. Not CMP-specific, but its author verified the HBM temperature readout on a CMP 170HX. Author: GitHub DingoOz, MIT, Rust. Repository: https://github.com/DingoOz/llm-visuals Source: @trustinter, 2026-10-17
Source: repo github.com/DingoOz/llm-visuals by DingoOz on GitHub, posted by @trustinter, 2026-10-17.
Install and run
Prebuilt binaries for Linux, Windows and macOS (x86-64 and ARM64) are on the GitHub releases page. From source, with Rust 1.75 or newer:
git clone https://github.com/DingoOz/llm-visuals && cd llm-visuals
cargo run --release -- --demo # no server needed
cargo run --release # auto-detects the local server
Source: repo github.com/DingoOz/llm-visuals by DingoOz on GitHub, posted by @trustinter, 2026-10-17.
What it shows that matters on a 170HX
- HBM temperature through NVML field value 82 (
temperature.memoryon the nvidia-smi fallback), next to the die temperature. It raises a "memory hot" alert when the HBM stays at or above the driver's memory max-operating temperature (95 C if the driver names none) for 3 seconds. Thresholds people use are on Monitoring. - Throttle reasons as badges: THERM (hardware thermal), BRAKE (hardware power brake, an external signal such as PWRBRK#), HW, PWR (software power cap), SYNC, APP.
- Energy: total board power over all cards, tokens per joule, Wh per 1,000 tokens.
- vLLM through its standard
/metrics, which the community's vLLM forks keep: decode, prefill without cached prompt tokens, prefix-cache hit rate, and the VRAM split from the served model and the server's command line.
Source: repo github.com/DingoOz/llm-visuals by DingoOz on GitHub, posted by @trustinter, 2026-10-17.
Caveats
- A general dashboard; the 170HX appears only as the card the memory temperature path was verified on.
- MoE expert routing needs a llama.cpp patch shipped in the repository; without it the expert view is a visual stand-in.
- Very active; note the version when you report numbers from it.
Source: repo github.com/DingoOz/llm-visuals by DingoOz on GitHub, posted by @trustinter, 2026-10-17.