The local-AI hardware index.
Aggregated numbers from real silicon, Llama, Mistral, DeepSeek, Qwen, SDXL, Whisper. Sortable, filterable, with $/tok·s⁻¹ and perf/watt so you can decide what to actually buy.
Where these numbers come from: These benchmarks are aggregated from llama.cpp community runs, vendor-published numbers, vLLM logs, and MLPerf submissions. MyAIHardware does not yet operate an in-house benchmark lab, that's coming Q3 2026. See per-row source notes and our full methodology.
Reigning champion · Llama 3 8B Q4
AMD · Q4_K_M · 192 GB VRAM · 750 W TDP
Filter by silicon class, quantization, price, and VRAM to find your perfect setup.
Llama 3 8B (Q4_K_M) · sorted by tokens / sec
Each dot is a device. Top-left is best value (cheap + fast).
Click any row for detailed breakdown. Click column headers to sort.
| # | Device | Verif | Buy | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | AMD Instinct MI300X 192GBDatacenter GPU AMD·Q4_K_M·4K ctx | 6 | 920tok/s | 192 GB | 750 W | $15k | 1.23 tok/s/W | $0.17/M | N/A | |
| 2 | NVIDIA GeForce RTX 5090 32GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 720tok/s | 32 GB | 575 W | $2.0k | 1.25 tok/s/W | $0.03/M | N/A | |
| 3 | NVIDIA GeForce RTX 4090 24GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 540tok/s | 24 GB | 450 W | $1.6k | 1.20 tok/s/W | $0.03/M | N/A | |
| #4 | NVIDIA B200 192GBDatacenter GPU NVIDIA·Q4_K_M·128K ctx | 6 | 360tok/s | 192 GB | 1000 W | $40k | 0.36 tok/s/W | $0.0012/k | N/A | |
| #5 | AWS Trainium2ASIC AWS·Q4_K_M·8K ctx | 6 | 215tok/s | 96 GB | 500 W | $28k | 0.43 tok/s/W | $0.0014/k | N/A | |
| #6 | NVIDIA H200 141GBDatacenter GPU NVIDIA·Q4_K_M·128K ctx | 6 | 215tok/s | 141 GB | 700 W | $30k | 0.31 tok/s/W | $0.0015/k | N/A | |
| #7 | NVIDIA GeForce RTX 5090 32GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 182tok/s σ 5.6 | 32 GB | 575 W | $2.0k | 0.32 tok/s/W | $0.12/M | Amazon | |
| #8 | NVIDIA H100 SXM5 80GBDatacenter GPU NVIDIA·Q4_K_M·128K ctx | 6 | 178tok/s | 80 GB | 700 W | $25k | 0.25 tok/s/W | $0.0015/k | N/A | |
| #9 | NVIDIA GeForce RTX 5080 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 165tok/s σ 4.8 | 16 GB | 360 W | $999 | 0.46 tok/s/W | $0.06/M | N/A | |
| #10 | NVIDIA GeForce RTX 5090 32GBConsumer GPU NVIDIA·Q4_K_M·32K ctx | 6 | 162tok/s | 32 GB | 575 W | $2.0k | 0.28 tok/s/W | $0.13/M | N/A | |
| #11 | NVIDIA GeForce RTX 4090 24GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 132tok/s σ 3.8 | 24 GB | 450 W | $1.6k | 0.29 tok/s/W | $0.13/M | Amazon | |
| #12 | NVIDIA GeForce RTX 5070 Ti 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 128tok/s | 16 GB | 300 W | $749 | 0.43 tok/s/W | $0.06/M | N/A | |
| #13 | AMD Instinct MI300X 192GBDatacenter GPU AMD·Q4_K_M·128K ctx | 6 | 122tok/s | 192 GB | 750 W | $15k | 0.16 tok/s/W | $0.0013/k | N/A | |
| #14 | NVIDIA GeForce RTX 4090 24GBConsumer GPU NVIDIA·Q4_K_M·32K ctx | 6 | 112tok/s | 24 GB | 450 W | $1.6k | 0.25 tok/s/W | $0.15/M | N/A | |
| #15 | NVIDIA GeForce RTX 4080 Super 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 102tok/s σ 3.4 | 16 GB | 320 W | $999 | 0.32 tok/s/W | $0.10/M | N/A | |
| #16 | Apple M3 Ultra (80c GPU, 512GB)Apple Silicon Apple·Q4_K_M·8K ctx | 6 | 102tok/s | 512 GB | 270 W | $9.5k | 0.38 tok/s/W | $0.98/M | N/A | |
| #17 | AMD Radeon RX 7900 XTX 24GBConsumer GPU AMD·Q4_K_M·4K ctx | 6 | 95tok/s | 24 GB | 355 W | $999 | 0.27 tok/s/W | $0.11/M | Amazon | |
| #18 | NVIDIA GeForce RTX 5070 12GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 92tok/s | 12 GB | 250 W | $549 | 0.37 tok/s/W | $0.06/M | N/A | |
| #19 | NVIDIA GeForce RTX 3090 24GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 88tok/s | 24 GB | 350 W | $1.5k | 0.25 tok/s/W | $0.18/M | N/A | |
| #20 | NVIDIA GeForce RTX 4070 Ti 12GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 78tok/s | 12 GB | 285 W | $799 | 0.27 tok/s/W | $0.11/M | N/A | |
| #21 | NVIDIA GeForce RTX 5060 Ti 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 72tok/s | 16 GB | 180 W | $449 | 0.40 tok/s/W | $0.07/M | N/A | |
| #22 | AMD Radeon RX 7800 XT 16GBConsumer GPU AMD·Q4_K_M·4K ctx | 6 | 64tok/s | 16 GB | 263 W | $499 | 0.24 tok/s/W | $0.08/M | Amazon | |
| #23 | NVIDIA A40 48GBPro GPU NVIDIA·Q4_K_M·4K ctx | 6 | 64tok/s | 48 GB | 300 W | $5.5k | 0.21 tok/s/W | $0.91/M | N/A | |
| #24 | NVIDIA GeForce RTX 4070 12GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 62tok/s | 12 GB | 200 W | $599 | 0.31 tok/s/W | $0.10/M | N/A | |
| #25 | NVIDIA RTX A5000 24GBPro GPU NVIDIA·Q4_K_M·4K ctx | 6 | 58tok/s | 24 GB | 230 W | $2.2k | 0.25 tok/s/W | $0.40/M | N/A | |
| #26 | NVIDIA GeForce RTX 5060 Ti 8GBConsumer GPU NVIDIA·Q4_K_M·2K ctx | 6 | 58tok/s | 8 GB | 165 W | $379 | 0.35 tok/s/W | $0.07/M | N/A | |
| #27 | Apple M4 Max (40c GPU, 128GB)Apple Silicon Apple·Q4_K_M·8K ctx | 6 | 56tok/s σ 1.8 | 128 GB | 70 W | $4.7k | 0.80 tok/s/W | $0.89/M | N/A | |
| #28 | AMD Radeon RX 7700 XT 12GBConsumer GPU AMD·Q4_K_M·4K ctx | 6 | 52tok/s | 12 GB | 245 W | $449 | 0.21 tok/s/W | $0.09/M | N/A | |
| #29 | Apple M3 Max (40c GPU, 64GB)Apple Silicon Apple·Q4_K_M·8K ctx | 6 | 50tok/s | 64 GB | 65 W | $3.5k | 0.77 tok/s/W | $0.74/M | N/A | |
| #30 | NVIDIA GeForce RTX 4060 Ti 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 48tok/s | 16 GB | 165 W | $499 | 0.29 tok/s/W | $0.11/M | N/A | |
| #31 | NVIDIA GeForce RTX 5060 8GBConsumer GPU NVIDIA·Q4_K_M·2K ctx | 6 | 48tok/s | 8 GB | 145 W | $299 | 0.33 tok/s/W | $0.07/M | N/A | |
| #32 | Apple M4 Max (40c GPU, 128GB)Apple Silicon Apple·Q4_K_M·32K ctx | 6 | 48tok/s | 128 GB | 65 W | $4.7k | 0.74 tok/s/W | $0.0010/k | N/A | |
| #33 | NVIDIA GeForce RTX 3060 12GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 42tok/s | 12 GB | 170 W | $329 | 0.25 tok/s/W | $0.08/M | N/A | |
| #34 | NVIDIA RTX A4000 16GBPro GPU NVIDIA·Q4_K_M·4K ctx | 6 | 42tok/s | 16 GB | 140 W | $1.0k | 0.30 tok/s/W | $0.25/M | N/A | |
| #35 | NVIDIA GeForce RTX 4060 8GBConsumer GPU NVIDIA·Q4_K_M·2K ctx | 6 | 42tok/s | 8 GB | 115 W | $299 | 0.36 tok/s/W | $0.07/M | N/A | |
| #36 | Intel Arc B580 12GBConsumer GPU Intel·Q4_K_M·4K ctx | 6 | 38tok/s | 12 GB | 190 W | $249 | 0.20 tok/s/W | $0.07/M | N/A | |
| #37 | Apple M2 Max (38c GPU, 96GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 38tok/s | 96 GB | 60 W | $3.3k | 0.63 tok/s/W | $0.92/M | N/A | |
| #38 | Apple M4 Pro (20c GPU, 48GB)Apple Silicon Apple·Q4_K_M·8K ctx | 6 | 38tok/s | 48 GB | 50 W | $2.5k | 0.76 tok/s/W | $0.69/M | N/A | |
| #39 | Apple M3 Ultra (80c GPU, 512GB)Apple Silicon Apple·Q4_K_M·128K ctx | 6 | 38tok/s | 512 GB | 100 W | $10.0k | 0.38 tok/s/W | $0.0028/k | N/A | |
| #40 | Intel Arc A770 16GBConsumer GPU Intel·Q4_K_M·4K ctx | 6 | 36tok/s | 16 GB | 225 W | $329 | 0.16 tok/s/W | $0.10/M | N/A | |
| #41 | NVIDIA GeForce RTX 4060 8GBConsumer GPU NVIDIA·Q4_K_M·2K ctx | 6 | 32tok/s | 8 GB | 115 W | $299 | 0.28 tok/s/W | $0.10/M | N/A | |
| #42 | Apple Mac mini M4 Pro 24GBApple Silicon Apple·Q4_K_M·8K ctx | 6 | 32tok/s | 24 GB | 35 W | $1.4k | 0.91 tok/s/W | $0.46/M | N/A | |
| #43 | AMD Radeon RX 7600 8GBConsumer GPU AMD·Q4_K_M·2K ctx | 6 | 28tok/s | 8 GB | 165 W | $269 | 0.17 tok/s/W | $0.10/M | N/A | |
| #44 | AMD Ryzen AI Max+ 395 (Strix Halo, 96GB)NPU AMD·Q4_K_M·8K ctx | 6 | 28tok/s | 96 GB | 120 W | $2.2k | 0.23 tok/s/W | $0.83/M | N/A | |
| #45 | Intel Arc A750 8GBConsumer GPU Intel·Q4_K_M·2K ctx | 6 | 24tok/s | 8 GB | 225 W | $249 | 0.11 tok/s/W | $0.11/M | N/A | |
| #46 | NVIDIA Jetson AGX Orin 64GBEdge NVIDIA·Q4_K_M·4K ctx | 6 | 22tok/s | 64 GB | 60 W | $2.0k | 0.37 tok/s/W | $0.96/M | N/A | |
| #47 | Apple M3 Pro (18c GPU, 36GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 22tok/s | 36 GB | 50 W | $2.5k | 0.44 tok/s/W | $0.0012/k | N/A | |
| #48 | Apple M2 Pro (19c GPU, 32GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 20tok/s | 32 GB | 45 W | $2.0k | 0.44 tok/s/W | $0.0011/k | N/A | |
| #49 | Apple M4 (10c GPU, 24GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 18tok/s | 24 GB | 22 W | $1.6k | 0.82 tok/s/W | $0.94/M | N/A | |
| #50 | NVIDIA Jetson Orin NX 16GBEdge NVIDIA·Q4_K_M·4K ctx | 6 | 18tok/s | 16 GB | 25 W | $599 | 0.72 tok/s/W | $0.35/M | N/A | |
| #51 | Apple Mac mini M4 24GBApple Silicon Apple·Q4_K_M·8K ctx | 6 | 17tok/s | 24 GB | 22 W | $999 | 0.77 tok/s/W | $0.62/M | N/A | |
| #52 | Apple Mac mini M4 16GBApple Silicon Apple·Q4_K_M·4K ctx | 6 | 16tok/s | 16 GB | 18 W | $599 | 0.89 tok/s/W | $0.40/M | N/A | |
| #53 | NVIDIA Jetson Orin Nano Super 8GBEdge NVIDIA·Q4_K_M·2K ctx | 6 | 14tok/s | 8 GB | 25 W | $249 | 0.56 tok/s/W | $0.19/M | N/A | |
| #54 | Intel Xeon 6980P (128c Granite Rapids)CPU-only Intel·Q4_K_M·4K ctx | 6 | 14tok/s | , | 500 W | $18k | 0.03 tok/s/W | $0.0134/k | N/A | |
| #55 | AMD Threadripper PRO 7995WX (96-core)CPU-only AMD·Q4_K_M·4K ctx | 6 | 12tok/s | , | 350 W | $10.0k | 0.03 tok/s/W | $0.0088/k | N/A | |
| #56 | Qualcomm Snapdragon X Elite (X1E-84-100)NPU Qualcomm·INT4·4K ctx | 6 | 12tok/s | , | 23 W | $1.2k | 0.52 tok/s/W | $0.0011/k | Amazon | |
| #57 | Qualcomm Snapdragon X Elite (X1E-84-100)NPU Qualcomm·INT4·2K ctx | 6 | 11tok/s | , | 23 W | $1.2k | 0.48 tok/s/W | $0.0012/k | Amazon | |
| #58 | Intel Core Ultra 9 288V (Lunar Lake) NPUNPU Intel·INT4·2K ctx | 6 | 9tok/s | , | 17 W | $549 | 0.53 tok/s/W | $0.64/M | Amazon | |
| #59 | NVIDIA Jetson Orin Nano 8GBEdge NVIDIA·Q4_K_M·2K ctx | 6 | 8tok/s | 8 GB | 15 W | $499 | 0.53 tok/s/W | $0.66/M | Amazon | |
| #60 | AMD Ryzen AI 9 HX 370 NPUNPU AMD·INT4·2K ctx | 6 | 7tok/s | , | 28 W | $1.5k | 0.25 tok/s/W | $0.0023/k | N/A | |
| #61 | AMD Ryzen AI 9 365 NPUNPU AMD·INT4·2K ctx | 6 | 6tok/s | , | 28 W | $1.2k | 0.21 tok/s/W | $0.0021/k | N/A | |
| #62 | Intel Core Ultra 7 258V (Lunar Lake)NPU Intel·INT4·2K ctx | 6 | 6tok/s | , | 17 W | $1.3k | 0.32 tok/s/W | $0.0025/k | N/A | |
| #63 | Raspberry Pi 5 + AI HAT+ (Hailo-8)Edge Raspberry Pi·INT8·1K ctx | 6 | 1tok/s | 8 GB | 12 W | $150 | 0.10 tok/s/W | $0.0013/k | Amazon |
Quick reference for every benchmark we run, including what it actually measures.
We separate hardware speed from model quality. These upstream snapshots show what the broader model-eval world thinks is strong right now, while our local benchmark data shows what actually runs well on your machine.
Snapshot generated
May 28, 2026
Weekly automated ingest with source-level freshness metadata.
Open-weight shortlist
6 local-friendly picks
Filtered to open text models with published parameter sizes and plausible local fit.
Reference sources
3 feeds
OpenEvals, LM Arena community Elo, and LiveBench model judgments.
Best open models to run locally
Directional quality signal for open-weight models. We bias this slice toward text models with known parameter sizes so builders can map quality to hardware fit.
#1
microsoft/Phi-3-medium-4k-instruct
14B · est. 8.4 GB Q4
#2
Qwen/Qwen2-72B
73B · est. 43.6 GB Q4
#3
microsoft/Phi-3.5-mini-instruct
3.8B · est. 2.3 GB Q4
#4
internlm/internlm2_5-7b-chat
7.7B · est. 4.6 GB Q4
#5
microsoft/Phi-3-mini-4k-instruct
3.8B · est. 2.3 GB Q4
Community preference ceiling
Useful for the broad popularity and preference frontier. This is mostly hosted frontier-model context, not a local-fit recommender.
#1
claude-opus-4-6-thinking
anthropic · 27.5K votes
#2
claude-opus-4-6
anthropic · 29.2K votes
#3
gemini-3.5-flash
google · 5.9K votes
#4
claude-opus-4-7-thinking
anthropic · 12.9K votes
#5
gemini-3.1-pro-preview
google · 34.2K votes
Recent task-judged performance
Another external frontier reference. We surface it so readers can separate 'runs fast locally' from 'is judged strong on recent tasks'.
#1
claude-3-7-sonnet-20250219-base
1 task buckets · 29 evals
#2
claude-3-opus-20240229
1 task buckets · 29 evals
#3
o1-2024-12-17-high
1 task buckets · 29 evals
#4
claude-3-7-sonnet-20250219-thinking-64k
1 task buckets · 29 evals
#5
gemini-2.0-flash-thinking-exp-1219
1 task buckets · 29 evals
Every record ships with the workload, model, quantization, runtime notes, source, and test date we could verify from the cited run. Because this is a blended dataset rather than one locked lab harness, compare results within the same workload and verification tier.
This release combines cited public runs from llama.cpp, MLPerf, vendor disclosures, and community logs. We normalize the metadata and clearly tag verification status rather than pretending every row came from one in-house harness.
Each workload page shows a representative prompt set and runtime command so you can reproduce the class of test. Exact prompts, runtimes, and harness settings still depend on the cited source for each record.
When a model doesn't fit in VRAM we explicitly note partial CPU offload. Pure-GPU runs and offload runs are not combined in the same rank.
Schema lives in src/data/benchmarkDatabase.ts on GitHub. Every record carries a sourceNote and testedAt date. v1.3 is the current cut, and the downloadable API is generated from the same enriched dataset the app renders.
Vendor-published numbers keep their sourceNote and verification label so you can apply your own discount. We do not silently blur vendor claims, community runs, and lab-verified records into one confidence tier.
$/1k tok is derived from MSRP and observed throughput using the public formula shown here. Perf/W is tokens/sec divided by TDP. No hidden weights, no proprietary composite score.
Running an exotic setup, Strix Halo, 4×3090 in a frame, M3 Ultra at 512GB unified? We want the data. Submissions are reviewed and credited. Repeat contributors get early access to upcoming benchmark releases.
Weekly AI hardware news + deals
Cross-reference benchmarks with our hardware databases.