Ollama Hardware Requirements & Recommended Builds
Ollama makes local LLMs almost too easy. The hard part is picking hardware that actually keeps up. This is the definitive 2026 reference for what runs, how fast, and where the cliffs are, for every Ollama model size from a 3B phone-runnable chatbot to the 671B-parameter MoE monsters.
On this page
TL;DR, the cheat sheet
- • 8 GB VRAM: 7B–8B at Q4 only. Skip 13B.
- • 12 GB VRAM: Comfortable 8B, tight 13B at Q4.
- • 16 GB VRAM: Sweet spot for 13B Q4 + small 30B Q4 with KV trimming.
- • 24 GB VRAM: Full 30B Q4 + 70B Q3 with care.
- • 32 GB VRAM (RTX 5090): 30B Q5 / 70B Q3 comfortably.
- • 48 GB+ VRAM: 70B Q4 in one card.
- • Apple Silicon ≥ 64 GB: 70B Q4 works fine via unified memory.
- • Storage: Gen 4 NVMe, 2 TB+. Models are huge.
Top picks per budget tier
Hardware we have personally tested with Ollama for at least 30 days.
Mac Studio M3 Ultra (128 GB)
Apple
The single most painless way to run Llama 3.1 70B locally. 128 GB unified memory becomes 96 GB usable VRAM, and Ollama on Metal just works.
RTX 3090 + 64 GB DDR4 PC
Custom
Used RTX 3090 plus a Ryzen 5950X / 12700K board with 64 GB RAM gets you to 70B Q4 territory for under $1,500 in 2026.
Mac Mini M4 (24 GB)
Apple
A $999 Mac Mini M4 with 24 GB unified memory runs Llama 3.1 8B at Q4 effortlessly. The cheapest credible Ollama box you can buy new.
RTX 5090 desktop
NVIDIA
If you only want the highest tokens/sec for the models you actually run today, build around a 5090. Ollama 0.4+ uses the new Blackwell FP4 path.
VRAM by model size
Memory budgets are approximate and assume an 8k context window. KV cache scales linearly with context length, double the context, expect 15–25% more VRAM consumption.
| Size class | Examples | Q4_K_M | Q5_K_M | FP16 | RAM fallback | Use cases |
|---|---|---|---|---|---|---|
| 3B | Phi-3 Mini, Gemma 3B | 2.5 GB | 3.0 GB | 6 GB | 8 GB | Auto-complete, classifiers |
| 7B–8B | Llama 3.1 8B, Qwen 2.5 7B, Mistral 7B | 5 GB | 6 GB | 14 GB | 16 GB | Default chat assistant |
| 13B | Qwen 2.5 14B, Codestral | 9 GB | 11 GB | 26 GB | 24 GB | Better reasoning chat |
| 30B–34B | DeepSeek-R1-Distill-32B, Yi 34B, Qwen 32B | 20 GB | 24 GB | 62 GB | 48 GB | Reasoning, code |
| 70B | Llama 3.1 70B, Qwen 2.5 72B | 40 GB | 48 GB | 140 GB | 96 GB | Frontier-quality chat |
| MoE 100B+ | Mixtral 8x22B, DeepSeek V3 | ≥80 GB | ≥100 GB | ≥260 GB | 256 GB | Pro workstations |
Minimum vs recommended specs
Minimum (will run)
- • 8 GB system RAM
- • 4-core CPU with AVX2 (Intel Haswell or AMD Zen+)
- • 32 GB free storage (SATA SSD acceptable)
- • Any GPU with 6 GB+ VRAM or CPU-only
- • Linux, macOS, or Windows 10+
Expect: 3B–7B Q4 models at 5–25 tok/s on CPU. Painful but functional.
Recommended (daily driver)
- • 32–64 GB system RAM (DDR5)
- • 8-core Zen 4 / Core 14th gen+ / Apple Silicon
- • 2 TB Gen 4 NVMe (models eat space)
- • 16–24 GB VRAM GPU OR ≥ 48 GB Apple unified memory
- • Linux LTS or macOS 14+
Expect: 8B–13B Q5 at 60+ tok/s; 30B Q4 at 35+ tok/s; 70B Q4 borderline.
Enthusiast / Pro
- • 128 GB system RAM ECC
- • Threadripper Pro / Apple M3 Ultra
- • 4 TB Gen 5 NVMe + 24 TB NAS
- • 32 GB+ VRAM (RTX 5090) or dual 24 GB (RTX 3090/4090 ×2)
- • Linux server LTS, Docker, IPMI
- • Dedicated 20 A circuit
Expect: 70B Q4 at 30+ tok/s; multi-model orchestration; fine-tuning LoRAs.
Platform support matrix
Where Ollama actually works well in May 2026.
| Platform | Maturity | Notes |
|---|---|---|
| NVIDIA + Linux | Excellent | CUDA + cuBLAS. Default path. All quants, latest features. |
| NVIDIA + Windows | Excellent | Installer ships CUDA bundled. WSL2 also works. |
| Apple Silicon (M1–M4) | Excellent | Metal backend. Unified memory acts as VRAM. |
| AMD ROCm + Linux | Good (RDNA3+) | RX 7700 XT and newer, MI300, Radeon Pro. |
| AMD Vulkan | Good | Cross-vendor fallback. Slower than CUDA / ROCm. |
| Intel Arc / Xe | Beta | SYCL backend. Works for 7B–13B at usable speed. |
| CPU-only | Functional | AVX-512 / ARM NEON kernels. Slow but works. |
NVIDIA
CUDA is the gold standard. Ollama auto-installs the right CUDA libs in the official installer; on Linux just have a recent NVIDIA driver. Works great in Docker via the NVIDIA Container Toolkit.
Apple Silicon
Metal backend, GGUF tensors mapped directly into unified memory. A 64 GB Mac Studio runs 70B Q4 with room to spare. M3 Ultra in 192 GB config will host even DeepSeek V3 671B MoE at usable speeds.
AMD
RDNA3 RX 7700 XT and newer plus all MI300/MI325X work on ROCm 6.2+ under Linux. Windows ROCm support remains preview. RX 7900 XTX is the value pick, 24 GB at consumer prices.
Tokens-per-second across hardware
Single-stream, batch 1, 4k context. Measured on Ollama 0.6.4 (May 2026).
Getting started in five commands
Ollama on a fresh Linux box, NVIDIA GPU. Same flow on macOS, swap the installer line for the Mac DMG.
# Linux & macOS
curl -fsSL https://ollama.com/install.sh | sh
# Windows users: download the installer from
# https://ollama.com/download/windowsollama pull llama3.1:8b-instruct-q4_K_M
# 4.7 GB download
ollama pull llama3.1:70b-instruct-q4_K_M
# 40 GB download, go get coffeeollama run llama3.1:8b-instruct-q4_K_M
>>> Hello, who are you?ollama ps
# NAME SIZE PROCESSOR UNTIL
# llama3.1:8b-instruct-q4_K_M 6.2 GB 100% GPU 4 minutes from now
nvidia-smi --query-gpu=memory.used --format=csv# Keep 2 models warm at once
export OLLAMA_MAX_LOADED_MODELS=2
# Larger context window (defaults to 2048)
export OLLAMA_CONTEXT_LENGTH=8192
# Bind to LAN so other devices can use the server
export OLLAMA_HOST=0.0.0.0:11434
systemctl restart ollama # Linux
launchctl unload + load # macOSStorage planning
Ollama stores models in ~/.ollama/models (or /usr/share/ollama/.ollama on systemd hosts). A daily-driver setup with one 8B, one 13B coder, and one 30B reasoning model consumes roughly 30 GB. Plan on 200–500 GB if you collect models. Use NVMe, cold-load time on a SATA SSD turns Ollama into a coffee break.
Frequently asked questions
What is the minimum hardware to run Ollama?
Any 64-bit machine with 8 GB of RAM can run a 3B Q4 model on CPU at usable speeds. The honest minimum for a daily-driver setup is 16 GB RAM and a GPU with 8 GB VRAM, or an Apple Silicon Mac with 16 GB unified memory.
How much RAM and VRAM do I need for Llama 3.1 70B in Ollama?
On a CUDA GPU you want ~40 GB VRAM for Q4_K_M with an 8k context. On Apple Silicon, ~48 GB of unified memory is the comfortable floor, ~64 GB is comfortable, and 128 GB lets you run 70B + a second model simultaneously.
Does Ollama use the GPU automatically?
Yes, on NVIDIA (CUDA), Apple Silicon (Metal), and AMD (ROCm where supported). Ollama auto-detects available accelerators and offloads as many layers as VRAM allows. The remaining layers run on CPU.
Is Apple Silicon better than NVIDIA for Ollama?
It depends on the workload. NVIDIA is faster per-token on small-context inference. Apple Silicon wins on VRAM capacity per dollar (unified memory) and is silent and low-power. For models bigger than 30B, an M-series Mac with 64 GB+ unified memory frequently beats a 24 GB NVIDIA card.
Does Ollama work on AMD GPUs?
Yes, on a supported subset of RDNA3/CDNA cards via ROCm on Linux. RX 7900 XT / XTX, MI300, and newer Radeon Pro cards work well. Older RDNA2 cards are inconsistent.
How fast is the SSD requirement for Ollama?
Cold-load time is dominated by sequential read. A Gen 3 NVMe (~3 GB/s) loads a 40 GB 70B model in ~15 seconds. A Gen 4 (~7 GB/s) cuts that to ~7 seconds. A SATA SSD (~550 MB/s) takes over a minute. Use NVMe.
Can I run multiple Ollama models at once?
Yes, but each loaded model consumes VRAM independently. With OLLAMA_MAX_LOADED_MODELS you can keep N models warm. Plan on ~5 GB VRAM for an 8B Q4 plus ~40 GB for a 70B Q4 if you want both resident.
What is the difference between Q4_K_M and Q5_K_M?
Q4_K_M uses ~4.5 bits/weight on average; Q5_K_M uses ~5.5 bits/weight. Q5 produces slightly better perplexity (~1-2% lower) at the cost of ~25% more VRAM and ~10% slower inference. Q4_K_M is the default sweet spot.
Why does Ollama get slow when context fills up?
KV cache scales linearly with context length. A 70B model at 8k context uses ~6 GB of additional VRAM for the cache; at 32k context it uses ~24 GB. Once you spill out of VRAM into RAM, generation throughput drops 4–10×.
Does Ollama have a CPU-only mode?
Yes. With no GPU detected, Ollama falls back to a fully CPU inference path using llama.cpp's GGUF kernels. A 7B Q4 runs at 6–15 tok/s on a Zen 4 or Apple Silicon CPU; a 70B Q4 runs at 1–3 tok/s.
Stay Ahead of the AI Curve
Get weekly AI hardware news, benchmark updates, and deals in your inbox. Founding-subscriber list, be one of the first.
Affiliate disclosure: As an Amazon Associate, MyAIHardware.com earns from qualifying purchases. Our Ollama hardware tests are conducted independently before any affiliate partnership is considered.