Benchmark / AI PCs & local LLMs
AI PC benchmark: local-LLM speed and memory (2026)
Token speed ≈ memory bandwidth ÷ model size; unified-memory capacity decides which models fit; NPU TOPS is largely marketing. Here is the comparison that actually predicts local-LLM performance — with sources and honest caveats.
By Evan Cole · Last updated July 17, 2026
Local-LLM speed is easy to predict once you know the one rule the spec sheets bury: token-generation speed is roughly memory bandwidth divided by the size of the model in memory (real output lands around 50–80% of that ceiling). Unified-memory capacity decides which models you can load at all; bandwidth decides how fast they run. The NPU “TOPS” number the boxes advertise is close to irrelevant for large-model text generation — mainstream runtimes (llama.cpp, Ollama, MLX) use the GPU, not the NPU, and decode is bandwidth-bound, not arithmetic-bound.
So this table leads with memory and bandwidth. The tokens-per-second column is drawn from independent testers using different models, quantisations and frameworks — treat it as directional, not a single controlled benchmark. Figures are aggregated from independent inference testing, cited by source.
Local-LLM comparison table
| Machine | Silicon | Unified mem (GB) | Bandwidth (GB/s) | NPU TOPS | Max model @ Q4 | Measured tok/s (independent) |
|---|---|---|---|---|---|---|
| Apple Mac Studio (M3 Ultra) | Apple M3 Ultra | 256 | 819 | 32-core NE | ~405B Q4 (up to 512 GB config) | ~16.5 tok/s (70B, MLX) |
| Apple Mac Studio (M4 Max) | Apple M4 Max | 128 | 546 | 38 | 70B dense / ~100B MoE Q4 | ~60 tok/s (120B MoE); ~7 tok/s (70B) |
| Apple Mac mini (M4 Pro) | Apple M4 Pro | 48 | 273 | 38 | 70B Q4 tight (32B comfortable) | ~11 tok/s (34B) |
| ASUS Ascent GX10 | NVIDIA GB10 | 128 | 273 | GPU ~1 PFLOP FP4 | ~200B MoE / 70B dense Q4 | ~2.7 tok/s (70B); 11.7 (120B MoE) |
| Beelink GTR9 Pro | Ryzen AI Max+ 395 | 128 | 256 | 50 | ~120B MoE / 70B dense Q4 | ~5 tok/s (70B); 72 (30B MoE) |
| Minisforum MS-S1 Max | Ryzen AI Max+ 395 | 64 | 256 | 50 | 70B Q4 | ~5 tok/s (70B) |
| Apple Mac mini (M4) | Apple M4 | 24 | 120 | 38 | ~14B (7–8B comfortable) | 42 tok/s (8B) |
Sorted fastest-bandwidth first. Prices and Amazon sellers on these niche machines are volatile and often reseller-inflated — re-check before buying. Full picks, verdicts and pros/cons are in our best AI PC for local LLMs guide.
How to read this
Bandwidth is the speedometer. On a Llama-3 70B dense model, the ~256–273 GB/s machines (NVIDIA GB10, Ryzen AI Max, Mac mini M4 Pro) cluster around 3–5 tokens/second; the Mac Studio M4 Max at 546 GB/s is markedly faster; and only the Apple-direct M3 Ultra at 819 GB/s reaches the ~16–20+ tok/s range. If you want responsiveness on big models, bandwidth is what you're buying.
Unified memory is the door. It decides which models load at all. As rough footprints: Q4 ≈ 0.55–0.6 GB per billion parameters, Q8 ≈ 1.1 GB per billion, then add 10–20% for context/KV-cache:
- 24 GB → 7B–14B comfortably; not 70B.
- 48 GB → 32B comfortably; 70B Q4 tight.
- 64 GB → 70B Q4 comfortably.
- 128 GB → 70B at Q8, or a ~120B mixture-of-experts model at Q4 (but a 235B MoE ≈ 132 GB still won't fit).
- 256 GB → ~405B at Q4 tight (M3 Ultra territory).
Ignore the TOPS race. “50 TOPS,” “48 TOPS,” or platform totals over 100 TOPS describe the NPU, which helps with lightweight on-device features — not with how fast a 70B model answers. Judge a local-AI machine on the two columns above, not the marketing number.
Sources & method
Memory and bandwidth figures are from manufacturer specifications (NVIDIA DGX Spark documentation for GB10; Apple's Mac Studio and Mac mini spec pages; Framework, Corsair and HP spec pages for Ryzen AI Max). Tokens-per-second figures are independent third-party measurements (ServeTheHome, Level1Techs, LMSYS, MLX community results and MacRumors), gathered under differing quants and frameworks — presented as ranges, not one apples-to-apples chart. This page was produced with AI assistance as part of VerdictBits's research workflow; every spec was cross-checked against a primary source before publication, and figures we could not independently verify are flagged in the guide.
Related buyer's guides
All reviews →Buyer's guide / AI PCs
LLM VRAM calculator
Enter a model size, quantization and context length to estimate the VRAM or unified memory an LLM needs to run locally — and instantly see which 2026 machines can hold it.
Use the calculator →
Buyer's guide / AI PCs
How to run LLMs locally: the 2026 hardware guide
How much VRAM or unified memory you need to run 7B-120B models locally, the capacity-vs-bandwidth tradeoff, and the 2026 hardware that hits each point on the curve — GB10, Ryzen AI Max, Apple Silicon and RTX GPUs.
Read the guide →
Buyer's guide / AI PCs
Mac vs PC for running local LLMs (2026)
It's not Mac vs PC — it's capacity vs speed. A Mac's unified memory fits models a PC GPU can't hold; a PC GPU runs the models that fit far faster. A sourced head-to-head plus which to buy.
Read the comparison →
Buyer's guide / AI PCs
The best AI PC for running LLMs locally in 2026
Six machines for running large language models on your own hardware — from a $799 Mac mini on-ramp to an NVIDIA GB10 supercomputer — ranked by what actually matters: unified-memory capacity and bandwidth, not NPU TOPS.
6 picks →
Buyer's guide / AI PCs
Best CPU for AI and local LLMs (2026)
The honest version: for pure GPU inference the CPU barely matters — so this guide is organised by when it does. Consumer value, the Intel option, unified-memory APUs that hold models a GPU can't, and the workstation platform you only need for multi-GPU rigs.
6 picks →
Buyer's guide / PC builds
Best gaming PC under $1,500 in 2026
The honest $1,500 build: a 16GB RX 9060 XT, Ryzen 5 and 32GB DDR5 for strong 1080p-ultra and entry-1440p gaming — with the real shortage-adjusted total, where every dollar goes, and whether a prebuilt beats it this year.
See the build →