llm workload · 98 runs on record
Mistral 7B (Q4)
Mistral 7B Instruct v0.3, Q4_K_M, batch 1, 4K context.
Primary metric: Tokens / sec (tok/s)
Reference prompts
Representative prompts for this workload. Exact prompts and harness settings still depend on the cited source for each record.
- Prompt 1
Explain the relationship between attention heads and KV cache memory usage for a 7B-parameter transformer at 4K context.
- Prompt 2
Write a 60-word product description for a mid-range AI workstation: 1× RTX 4090, 64GB DDR5, 2TB NVMe. Highlight one trade-off.
- Prompt 3
List five concrete differences between INT4 weight-only quantization (Q4_K_M) and INT8 quantization (Q8_0) for inference.
- Prompt 4
I have 12GB of VRAM and want to run a coding assistant locally. What model size and quantization fit, with what context length?
- Prompt 5
Summarize the difference between greedy decoding and nucleus sampling in 3 short bullet points.
Reference runtime command
A representative invocation for reproducing this workload class. Source-specific runs may use adjacent runtimes unless the record says otherwise.
shell
llama-server -m mistral-7b-instruct-v0.3.Q4_K_M.gguf -c 4096 -ngl 999 --seed 42Single batch. The Mistral 7B numbers should track Llama 3 8B closely on compute-bound devices.
Full leaderboard
Every record for Mistral 7B.
Top 10 leaderboard
Mistral 7B (Q4) · sorted by tokens / sec
98 entries
Value frontier, MSRP vs tokens / sec
Each dot is a device. Top-left is best value (cheap + fast).
Full leaderboard
Click any row for detailed breakdown. Click column headers to sort.
| # | Device | Verif | Buy | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Cerebras WSE-3ASIC Cerebras·FP16·4K ctx | 6 | 1.9ktok/s | 44000 GB | 23000 W | $2.00M | 0.08 tok/s/W | $0.0114/k | Amazon | |
| 2 | Cerebras WSE-3 (CS-3)ASIC Cerebras·FP16·8K ctx | 6 | 1.6ktok/s | 44 GB | 23000 W | $2.50M | 0.07 tok/s/W | $0.0163/k | Amazon | |
| 3 | Groq LPU (cloud)ASIC Groq·FP16·8K ctx | 6 | 1.3ktok/s | 230 GB | 350 W | $20k | 3.66 tok/s/W | $0.17/M | Amazon | |
| #4 | Groq LPU Inference EngineASIC Groq·FP8·4K ctx | 6 | 750tok/s | 230 GB | 215 W | $20k | 3.49 tok/s/W | $0.28/M | Amazon | |
| #5 | Groq LPU (per chip)ASIC Groq·FP16·8K ctx | 6 | 540tok/s | 0.23 GB | 215 W | $20k | 2.51 tok/s/W | $0.39/M | Amazon | |
| #6 | NVIDIA B200 192GBDatacenter GPU NVIDIA·Q4_K_M·4K ctx | 6 | 415tok/s | 192 GB | 1000 W | $40k | 0.41 tok/s/W | $0.0010/k | Amazon | |
| #7 | Google TPU v5p (Trillium)ASIC Google·INT8·4K ctx | 6 | 410tok/s | 95 GB | 300 W | $0 | 1.37 tok/s/W | $0.00/M | Amazon | |
| #8 | NVIDIA H200 141GBDatacenter GPU NVIDIA·Q4_K_M·8K ctx | 6 | 305tok/s | 141 GB | 700 W | $30k | 0.44 tok/s/W | $0.0010/k | Amazon | |
| #9 | NVIDIA GH200 480GBDatacenter GPU NVIDIA·Q4_K_M·4K ctx | 6 | 305tok/s | 144 GB | 1000 W | $45k | 0.30 tok/s/W | $0.0016/k | Amazon | |
| #10 | NVIDIA H100 SXM5 80GBDatacenter GPU NVIDIA·Q4_K_M·16K ctx | 6 | 295tok/s | 80 GB | 700 W | $25k | 0.42 tok/s/W | $0.90/M | Amazon | |
| #11 | AWS Trainium2 Trn2ASIC AWS·INT8·4K ctx | 6 | 280tok/s | 96 GB | 350 W | $0 | 0.80 tok/s/W | $0.00/M | Amazon | |
| #12 | AMD Instinct MI300X 192GBDatacenter GPU AMD·Q4_K_M·4K ctx | 6 | 268tok/s | 192 GB | 750 W | $18k | 0.36 tok/s/W | $0.71/M | Amazon | |
| #13 | NVIDIA H200 141GBDatacenter GPU NVIDIA·Q4_K_M·4K ctx | 6 | 268tok/s | 141 GB | 700 W | $30k | 0.38 tok/s/W | $0.0012/k | Amazon | |
| #14 | NVIDIA H100 SXM5 80GBDatacenter GPU NVIDIA·Q4_K_M·4K ctx | 6 | 215tok/s | 80 GB | 700 W | $25k | 0.31 tok/s/W | $0.0012/k | Amazon | |
| #15 | NVIDIA GeForce RTX 5090 32GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 198tok/s | 32 GB | 575 W | $2.0k | 0.34 tok/s/W | $0.11/M | Amazon | |
| #16 | AMD Instinct MI300X 192GBDatacenter GPU AMD·Q4_K_M·4K ctx | 6 | 195tok/s | 192 GB | 750 W | $15k | 0.26 tok/s/W | $0.81/M | Amazon | |
| #17 | Google TPU v5eASIC Google·INT8·4K ctx | 6 | 195tok/s | 16 GB | 170 W | $0 | 1.15 tok/s/W | $0.00/M | Amazon | |
| #18 | NVIDIA GeForce RTX 5090 32GBConsumer GPU NVIDIA·Q4_K_M·8K ctx | 6 | 182tok/s | 32 GB | 575 W | $2.0k | 0.32 tok/s/W | $0.12/M | Amazon | |
| #19 | NVIDIA GeForce RTX 5090 32GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 178tok/s | 32 GB | 575 W | $2.0k | 0.31 tok/s/W | $0.12/M | Amazon | |
| #20 | NVIDIA A100 SXM4 80GBDatacenter GPU NVIDIA·Q4_K_M·4K ctx | 6 | 168tok/s | 80 GB | 400 W | $15k | 0.42 tok/s/W | $0.94/M | Amazon | |
| #21 | NVIDIA A100 SXM4 40GBDatacenter GPU NVIDIA·Q4_K_M·4K ctx | 6 | 162tok/s | 40 GB | 400 W | $10k | 0.41 tok/s/W | $0.65/M | Amazon | |
| #22 | NVIDIA GeForce RTX 4090 24GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 145tok/s | 24 GB | 450 W | $1.6k | 0.32 tok/s/W | $0.12/M | Amazon | |
| #23 | NVIDIA DGX Spark (Project DIGITS, 128GB)Datacenter GPU NVIDIA·Q4_K_M·4K ctx | 6 | 145tok/s | 128 GB | 240 W | $3.0k | 0.60 tok/s/W | $0.22/M | Amazon | |
| #24 | NVIDIA L40S 48GBDatacenter GPU NVIDIA·Q4_K_M·4K ctx | 6 | 142tok/s | 48 GB | 350 W | $7.8k | 0.41 tok/s/W | $0.58/M | Amazon | |
| #25 | NVIDIA GeForce RTX 4090 24GBConsumer GPU NVIDIA·Q4_K_M·8K ctx | 6 | 138tok/s | 24 GB | 450 W | $1.6k | 0.31 tok/s/W | $0.12/M | Amazon | |
| #26 | AMD Instinct MI250X 128GBDatacenter GPU AMD·Q4_K_M·4K ctx | 6 | 135tok/s | 128 GB | 560 W | $12k | 0.24 tok/s/W | $0.94/M | Amazon | |
| #27 | NVIDIA GeForce RTX 4090 24GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 128tok/s | 24 GB | 450 W | $1.6k | 0.28 tok/s/W | $0.13/M | Amazon | |
| #28 | NVIDIA RTX 6000 Ada 48GBPro GPU NVIDIA·Q4_K_M·4K ctx | 6 | 125tok/s | 48 GB | 300 W | $6.8k | 0.42 tok/s/W | $0.57/M | Amazon | |
| #29 | NVIDIA GeForce RTX 5080 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 122tok/s | 16 GB | 360 W | $999 | 0.34 tok/s/W | $0.09/M | Amazon | |
| #30 | NVIDIA GeForce RTX 4080 Super 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 118tok/s | 16 GB | 320 W | $999 | 0.37 tok/s/W | $0.09/M | Amazon | |
| #31 | NVIDIA GeForce RTX 4080 Super 16GBConsumer GPU NVIDIA·Q4_K_M·8K ctx | 6 | 108tok/s | 16 GB | 320 W | $999 | 0.34 tok/s/W | $0.10/M | Amazon | |
| #32 | NVIDIA GeForce RTX 5070 Ti 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 108tok/s | 16 GB | 300 W | $749 | 0.36 tok/s/W | $0.07/M | Amazon | |
| #33 | AMD Instinct MI210 64GBDatacenter GPU AMD·Q4_K_M·4K ctx | 6 | 105tok/s | 64 GB | 300 W | $7.5k | 0.35 tok/s/W | $0.76/M | Amazon | |
| #34 | NVIDIA GeForce RTX 4080 Super 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 102tok/s | 16 GB | 320 W | $999 | 0.32 tok/s/W | $0.10/M | Amazon | |
| #35 | AMD Radeon RX 7900 XTX 24GBConsumer GPU AMD·Q4_K_M·4K ctx | 6 | 95tok/s | 24 GB | 355 W | $999 | 0.27 tok/s/W | $0.11/M | Amazon | |
| #36 | NVIDIA GeForce RTX 4070 Ti 12GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 89tok/s | 12 GB | 285 W | $799 | 0.31 tok/s/W | $0.10/M | Amazon | |
| #37 | AMD Radeon RX 7900 XTX 24GBConsumer GPU AMD·Q4_K_M·4K ctx | 6 | 88tok/s | 24 GB | 355 W | $999 | 0.25 tok/s/W | $0.12/M | Amazon | |
| #38 | NVIDIA GeForce RTX 5070 12GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 88tok/s | 12 GB | 250 W | $549 | 0.35 tok/s/W | $0.07/M | Amazon | |
| #39 | NVIDIA A40 48GBPro GPU NVIDIA·Q4_K_M·4K ctx | 6 | 88tok/s | 48 GB | 300 W | $5.5k | 0.29 tok/s/W | $0.66/M | Amazon | |
| #40 | NVIDIA GeForce RTX 5070 12GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 85tok/s | 12 GB | 250 W | $549 | 0.34 tok/s/W | $0.07/M | Amazon | |
| #41 | Apple M3 Ultra (80c GPU, 512GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 82tok/s | 512 GB | 100 W | $10.0k | 0.82 tok/s/W | $0.0013/k | Amazon | |
| #42 | NVIDIA GeForce RTX 3090 24GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 78tok/s | 24 GB | 350 W | $1.5k | 0.22 tok/s/W | $0.20/M | Amazon | |
| #43 | NVIDIA GeForce RTX 4070 12GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 76tok/s | 12 GB | 200 W | $599 | 0.38 tok/s/W | $0.08/M | Amazon | |
| #44 | NVIDIA GeForce RTX 5060 Ti 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 74tok/s | 16 GB | 180 W | $499 | 0.41 tok/s/W | $0.07/M | Amazon | |
| #45 | AMD Radeon RX 7800 XT 16GBConsumer GPU AMD·Q4_K_M·4K ctx | 6 | 72tok/s | 16 GB | 263 W | $499 | 0.27 tok/s/W | $0.07/M | Amazon | |
| #46 | NVIDIA GeForce RTX 5060 Ti 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 68tok/s | 16 GB | 180 W | $449 | 0.38 tok/s/W | $0.07/M | Amazon | |
| #47 | Apple M3 Ultra (80c GPU, 512GB)Apple Silicon Apple·Q4_K_M·16K ctx | 6 | 68tok/s | 512 GB | 80 W | $8.5k | 0.85 tok/s/W | $0.0013/k | Amazon | |
| #48 | Apple M4 Max (40c GPU, 128GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 64tok/s | 128 GB | 65 W | $4.7k | 0.98 tok/s/W | $0.78/M | Amazon | |
| #49 | NVIDIA GeForce RTX 5060 8GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 64tok/s | 8 GB | 145 W | $299 | 0.44 tok/s/W | $0.05/M | Amazon | |
| #50 | NVIDIA RTX A5000 24GBPro GPU NVIDIA·Q4_K_M·4K ctx | 6 | 64tok/s | 24 GB | 230 W | $2.5k | 0.28 tok/s/W | $0.41/M | Amazon | |
| #51 | Apple M2 Ultra (76c GPU, 192GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 60tok/s | 192 GB | 80 W | $7.0k | 0.75 tok/s/W | $0.0012/k | Amazon | |
| #52 | AMD Radeon RX 7700 XT 12GBConsumer GPU AMD·Q4_K_M·4K ctx | 6 | 58tok/s | 12 GB | 245 W | $449 | 0.24 tok/s/W | $0.08/M | Amazon | |
| #53 | AMD Radeon RX 7800 XT 16GBConsumer GPU AMD·Q4_K_M·4K ctx | 6 | 58tok/s | 16 GB | 263 W | $499 | 0.22 tok/s/W | $0.09/M | Amazon | |
| #54 | NVIDIA GeForce RTX 4060 Ti 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 56tok/s | 16 GB | 165 W | $499 | 0.34 tok/s/W | $0.09/M | Amazon | |
| #55 | NVIDIA GeForce RTX 4060 Ti 16GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 56tok/s | 16 GB | 165 W | $499 | 0.34 tok/s/W | $0.09/M | Amazon | |
| #56 | Apple M3 Max (40c GPU, 128GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 52tok/s | 128 GB | 60 W | $4.0k | 0.87 tok/s/W | $0.81/M | Amazon | |
| #57 | NVIDIA GeForce RTX 3060 12GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 48tok/s | 12 GB | 170 W | $329 | 0.28 tok/s/W | $0.07/M | Amazon | |
| #58 | NVIDIA GeForce RTX 4060 8GBConsumer GPU NVIDIA·Q4_K_M·2K ctx | 6 | 48tok/s | 8 GB | 115 W | $299 | 0.42 tok/s/W | $0.07/M | Amazon | |
| #59 | AMD Radeon RX 7700 XT 12GBConsumer GPU AMD·Q4_K_M·4K ctx | 6 | 48tok/s | 12 GB | 245 W | $449 | 0.20 tok/s/W | $0.10/M | Amazon | |
| #60 | NVIDIA RTX A4000 16GBPro GPU NVIDIA·Q4_K_M·4K ctx | 6 | 48tok/s | 16 GB | 140 W | $1.2k | 0.34 tok/s/W | $0.26/M | Amazon | |
| #61 | Apple M2 Max (38c GPU, 96GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 42tok/s | 96 GB | 60 W | $3.3k | 0.70 tok/s/W | $0.83/M | Amazon | |
| #62 | Apple M4 Pro (20c GPU, 48GB)Apple Silicon Apple·Q4_K_M·8K ctx | 6 | 42tok/s | 48 GB | 50 W | $2.5k | 0.84 tok/s/W | $0.63/M | Amazon | |
| #63 | Intel Arc B580 12GBConsumer GPU Intel·Q4_K_M·4K ctx | 6 | 42tok/s | 12 GB | 190 W | $249 | 0.22 tok/s/W | $0.06/M | Amazon | |
| #64 | NVIDIA GeForce RTX 4060 8GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 42tok/s | 8 GB | 115 W | $299 | 0.36 tok/s/W | $0.07/M | Amazon | |
| #65 | Apple M2 Max (38c GPU, 96GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 42tok/s | 96 GB | 50 W | $3.5k | 0.84 tok/s/W | $0.88/M | Amazon | |
| #66 | Intel Arc A770 16GBConsumer GPU Intel·Q4_K_M·4K ctx | 6 | 40tok/s | 16 GB | 225 W | $329 | 0.18 tok/s/W | $0.09/M | Amazon | |
| #67 | AMD Ryzen AI Max+ 395 (Strix Halo, 96GB)NPU AMD·Q4_K_M·4K ctx | 6 | 38tok/s | 96 GB | 120 W | $2.2k | 0.32 tok/s/W | $0.61/M | Amazon | |
| #68 | NVIDIA GeForce RTX 3060 12GBConsumer GPU NVIDIA·Q4_K_M·4K ctx | 6 | 38tok/s | 12 GB | 170 W | $329 | 0.22 tok/s/W | $0.09/M | Amazon | |
| #69 | Apple Mac mini M4 Pro 48GBApple Silicon Apple·Q4_K_M·4K ctx | 6 | 38tok/s | 48 GB | 35 W | $2.0k | 1.09 tok/s/W | $0.56/M | Amazon | |
| #70 | Apple Mac mini M4 Pro 24GBApple Silicon Apple·Q4_K_M·4K ctx | 6 | 36tok/s | 24 GB | 35 W | $1.4k | 1.03 tok/s/W | $0.41/M | Amazon | |
| #71 | Apple M4 Pro (20c GPU, 48GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 36tok/s | 48 GB | 35 W | $2.0k | 1.03 tok/s/W | $0.59/M | Amazon | |
| #72 | Apple Mac mini M4 Pro 24GBApple Silicon Apple·Q4_K_M·4K ctx | 6 | 35tok/s | 24 GB | 35 W | $1.4k | 1.00 tok/s/W | $0.42/M | Amazon | |
| #73 | AMD Ryzen AI Max+ 395 (Strix Halo, 96GB)NPU AMD·Q4_K_M·8K ctx | 6 | 32tok/s | 96 GB | 120 W | $2.2k | 0.27 tok/s/W | $0.73/M | Amazon | |
| #74 | Intel Arc A770 16GBConsumer GPU Intel·Q4_K_M·4K ctx | 6 | 32tok/s | 16 GB | 225 W | $349 | 0.14 tok/s/W | $0.12/M | Amazon | |
| #75 | Apple M3 Pro (18c GPU, 36GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 28tok/s | 36 GB | 30 W | $2.5k | 0.93 tok/s/W | $0.94/M | Amazon | |
| #76 | Apple M3 Pro (18c GPU, 36GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 25tok/s | 36 GB | 50 W | $2.5k | 0.50 tok/s/W | $0.0011/k | Amazon | |
| #77 | NVIDIA Jetson AGX Orin 64GBEdge NVIDIA·Q4_K_M·4K ctx | 6 | 25tok/s | 64 GB | 60 W | $2.0k | 0.42 tok/s/W | $0.85/M | Amazon | |
| #78 | NVIDIA Jetson AGX Orin 64GBEdge NVIDIA·Q4_K_M·4K ctx | 6 | 24tok/s | 64 GB | 60 W | $2.0k | 0.40 tok/s/W | $0.88/M | Amazon | |
| #79 | Apple M2 Pro (19c GPU, 32GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 24tok/s | 32 GB | 28 W | $2.0k | 0.86 tok/s/W | $0.88/M | Amazon | |
| #80 | Apple Mac mini M4 24GBApple Silicon Apple·Q4_K_M·4K ctx | 6 | 23tok/s | 24 GB | 22 W | $799 | 1.04 tok/s/W | $0.37/M | Amazon | |
| #81 | Qualcomm Snapdragon X Elite X1E-84-100NPU Qualcomm·Q4_K_M·4K ctx | 6 | 22tok/s | , | 23 W | $1.2k | 0.96 tok/s/W | $0.58/M | Amazon | |
| #82 | Apple M4 (10c GPU, 24GB)Apple Silicon Apple·Q4_K_M·4K ctx | 6 | 22tok/s | 24 GB | 22 W | $999 | 1.00 tok/s/W | $0.48/M | Amazon | |
| #83 | Apple Mac mini M4 16GBApple Silicon Apple·Q4_K_M·4K ctx | 6 | 20tok/s | 16 GB | 22 W | $599 | 0.91 tok/s/W | $0.32/M | Amazon | |
| #84 | Apple Mac mini M4 16GBApple Silicon Apple·Q4_K_M·4K ctx | 6 | 18tok/s | 16 GB | 18 W | $599 | 1.00 tok/s/W | $0.35/M | Amazon | |
| #85 | Intel Xeon 6980P (128 cores)CPU-only Intel·Q4_K_M·4K ctx | 6 | 18tok/s | , | 500 W | $18k | 0.04 tok/s/W | $0.0105/k | Amazon | |
| #86 | AMD Ryzen AI 9 HX 370 (Strix Point NPU 50 TOPS)NPU AMD·Q4_K_M·4K ctx | 6 | 16tok/s | , | 28 W | $1.5k | 0.57 tok/s/W | $0.99/M | Amazon | |
| #87 | Intel Xeon 6980P (128c Granite Rapids)CPU-only Intel·Q4_K_M·4K ctx | 6 | 15tok/s | , | 500 W | $18k | 0.03 tok/s/W | $0.0125/k | Amazon | |
| #88 | AMD Threadripper PRO 7995WX (96-core)CPU-only AMD·Q4_K_M·4K ctx | 6 | 14tok/s | , | 350 W | $10.0k | 0.04 tok/s/W | $0.0075/k | Amazon | |
| #89 | AMD Threadripper Pro 7995WX (96 cores)CPU-only AMD·Q4_K_M·4K ctx | 6 | 14tok/s | , | 350 W | $10.0k | 0.04 tok/s/W | $0.0075/k | Amazon | |
| #90 | Intel Core Ultra 258V (Lunar Lake NPU 48 TOPS)NPU Intel·Q4_K_M·4K ctx | 6 | 13tok/s | , | 17 W | $1.3k | 0.77 tok/s/W | $0.0011/k | Amazon | |
| #91 | Qualcomm Snapdragon X Elite (X1E-84-100)NPU Qualcomm·INT4·2K ctx | 6 | 12tok/s | , | 23 W | $1.2k | 0.52 tok/s/W | $0.0011/k | Amazon | |
| #92 | NVIDIA Jetson Orin NX 16GBEdge NVIDIA·Q4_K_M·4K ctx | 6 | 10tok/s | 16 GB | 25 W | $699 | 0.40 tok/s/W | $0.74/M | Amazon | |
| #93 | Intel Core Ultra 9 285KCPU-only Intel·Q4_K_M·4K ctx | 6 | 9tok/s | , | 250 W | $589 | 0.04 tok/s/W | $0.69/M | Amazon | |
| #94 | AMD Ryzen AI 9 HX 370 NPUNPU AMD·INT4·2K ctx | 6 | 8tok/s | , | 28 W | $1.5k | 0.29 tok/s/W | $0.0020/k | Amazon | |
| #95 | NVIDIA Jetson Orin Nano Super 8GBEdge NVIDIA·Q4_K_M·4K ctx | 6 | 8tok/s | 8 GB | 25 W | $249 | 0.32 tok/s/W | $0.33/M | Amazon | |
| #96 | Intel Core Ultra 9 285K NPUNPU Intel·INT4·2K ctx | 6 | 7tok/s | , | 15 W | $589 | 0.47 tok/s/W | $0.89/M | Amazon | |
| #97 | NVIDIA Jetson Orin Nano 8GBEdge NVIDIA·Q4_K_M·4K ctx | 6 | 6tok/s | 8 GB | 15 W | $499 | 0.37 tok/s/W | $0.96/M | Amazon | |
| #98 | Raspberry Pi 5 8GBEdge Raspberry Pi·Q4_K_M·2K ctx | 6 | 2tok/s | , | 12 W | $80 | 0.15 tok/s/W | $0.47/M | Amazon |
Sorted by Tokens / sec (high → low)Tip: $/1k tok is a 3-year amortization of MSRP at sustained throughput.
Cite this benchmark
Use this in your paper, blog post, or comparison table.
BibTeX
@misc{myaihardware_mistral-7b-q4_2026,
title = {MyAI Bench: Mistral 7B (Q4)},
author = {{MyAIHardware Contributors}},
year = {2026},
url = {https://www.myaihardware.com/benchmarks/workload/mistral-7b-q4},
note = {Version 1.3, accessed 2026-08-27}
}APA
MyAIHardware Contributors. (2026). MyAI Bench: Mistral 7B (Q4). MyAIHardware. Retrieved 2026-08-27, from https://www.myaihardware.com/benchmarks/workload/mistral-7b-q4
MLA
MyAIHardware Contributors. "MyAI Bench: Mistral 7B (Q4)." MyAIHardware, 2026, https://www.myaihardware.com/benchmarks/workload/mistral-7b-q4. Accessed 2026-08-27.
Plain text
MyAI Bench, Mistral 7B (Q4). MyAIHardware Contributors, 2026. Version 1.3. https://www.myaihardware.com/benchmarks/workload/mistral-7b-q4 (accessed 2026-08-27).