Software GuideUpdated 2026-05-20R1 · V3 · Coder

Run DeepSeek Locally: The Complete Hardware Guide

DeepSeek's family of open-weights models, R1 reasoning, V3 chat, Coder for programming , punches above its weight class against GPT-4 and Claude. Most of it runs comfortably on the hardware you can buy at retail in May 2026. This is the hardware-first guide to running every DeepSeek variant locally, with measured tok/s and concrete budget tiers.

Priya Raghavan · Local AI Lead Updated 2026-05-20 20 min read Based on public benchmarks

Meet the DeepSeek family

DeepSeek-R1

DeepSeek's reasoning model, chain-of-thought, math, code, deep analysis. The full R1 is a 671B MoE; the practical interest for local users is the distilled lineup (1.5B through 70B) which preserves most of R1's reasoning at small-model footprints.

DeepSeek V3 / V3.1

The chat model. 671B parameters in an MoE topology with 37B active per token. Outperforms GPT-4o on most chat benchmarks. Runs on Mac Studio 192 GB at low single-digit tok/s, comfortably on a multi-node rig.

DeepSeek-Coder-V2

The coding specialist. Lite (16B MoE) runs on consumer cards and competes with the older Claude / GPT in everyday code tasks. The full 236B is a serious lift but unmatched among open-weights coders.

Best hardware per budget

What we actually run DeepSeek on in our lab.

best budget

RTX 3090 24GB

Runs DeepSeek-Coder-V2 16B comfortably and DeepSeek-R1 distilled 14B / 32B at usable speeds. The entry point to serious DeepSeek work.

24 GB · 936 GB/s
24 GB VRAM at used prices
Plenty fast for distilled R1
Cannot hold the full R1 671B
$799Amazon
best value

RTX 4090 24GB

Roughly 2× the throughput of a 3090 on DeepSeek R1 distilled models with the same 24 GB envelope. Best per-dollar performance in May 2026.

24 GB · 1008 GB/s
Excellent throughput on distilled R1 variants
Fast on DeepSeek-V3 Q4 MoE expert load
Still capped at 24 GB
$1,599Amazon
best overall

RTX 5090 32GB

32 GB lets you keep DeepSeek-R1-Distill-32B at Q5 + an 8k context window resident with comfortable headroom. The sweet spot for one-GPU DeepSeek.

32 GB GDDR7 · 1792 GB/s
Holds distilled-32B at Q5 with context to spare
FP4 path on Blackwell suits DeepSeek-V3 MoE
Won't fit full 671B even at extreme quants
$1,999Amazon
premium

Mac Studio M3 Ultra 192 GB

192 GB unified memory makes the M3 Ultra one of the few single-machine hosts that can actually run DeepSeek-V3 at Q4 on consumer-tier hardware.

M3 Ultra · 192 GB unified
Massive VRAM pool
Quiet, low-power
Slower per-expert than CUDA
$7,499 Deals

VRAM per DeepSeek model

Approximate VRAM including KV cache for an 8k context. MoE models report full-weight footprint; with smart loaders, only active experts need to be resident, but the cold model still occupies that disk.

ModelParamsQ4_K_MQ5_K_MQ8_0Hardware noteUse case
DeepSeek-R1-Distill-Qwen-1.5B1.5B1.5 GB2 GB3 GBEven on iGPUEmbedded reasoning, mobile
DeepSeek-R1-Distill-Qwen-7B7B5 GB6 GB9 GBSmooth on 8GB cardsDay-to-day reasoning chat
DeepSeek-R1-Distill-Qwen-14B14B10 GB12 GB18 GBRTX 4070 Super ✅Solid reasoning at low cost
DeepSeek-R1-Distill-Qwen-32B32B20 GB24 GB36 GBRTX 4090 / 5090 sweet spotBest near-R1 quality at home
DeepSeek-R1-Distill-Llama-70B70B40 GB48 GB72 GB2× RTX 3090 / RTX 6000 AdaHighest-quality distilled R1
DeepSeek-Coder-V2-Lite16B MoE10 GB12 GB18 GBRTX 4070 / RX 7800 XTLocal coding agent
DeepSeek-Coder-V2236B MoE≥130 GB≥165 GB≥250 GBMac Studio 192 GB or 4× GPUFrontier coding model
DeepSeek-V3671B MoE≥370 GB≥460 GB≥700 GBMulti-node onlyFrontier chat at production scale
DeepSeek-V3.1 (chat)671B MoE≥370 GB≥460 GB≥700 GBMac M3 Ultra 192 GB barelyBest open chat in 2026

Measured throughput per hardware

Single-stream tokens-per-second on the DeepSeek model variant best suited to each hardware tier. Batch 1, 4k context, llama.cpp + Ollama.

015304560RTX 5090, Distill-32B Q5RTX 4090, Distill-32B Q42× RTX 3090, Distill-70B Q4RTX 3090, Distill-32B Q4M3 Ultra 192 GB, V3 Q4 MoEM4 Max, Distill-32B Q4RTX 4070 Super, Distill-14B Q4RTX 3060 12GB, Distill-7B Q4

Variant deep dives

DeepSeek-R1 distills, the reasoning sweet spot

The full DeepSeek-R1 671B is impractical on consumer hardware, but the official distilled variants, derived by training Qwen 2.5 and Llama 3 backbones on R1's chain-of-thought traces , punch dramatically above their weight on math, code, and multi-step reasoning evals. R1-Distill-Qwen-32B at Q5_K_M is, in our testing, the single best reasoning model that fits in a 24 GB consumer card.

Watch the <think> tags. R1 distills generate hidden chain-of-thought before producing the final answer; the resulting context is often 2–4× the user-visible response. KV cache balloons accordingly. If you regularly push the context window past 16k tokens, plan for ~25% additional VRAM overhead beyond the table above.

Recommended

R1-Distill-Qwen-32B Q5_K_M on RTX 5090 32 GB. ~48 tok/s, 16k context fits.

Avoid

R1 distills at Q2 / Q3, reasoning collapses fast under aggressive quantization.

DeepSeek V3 / V3.1, the open frontier chat

V3 is a 671B MoE with 37B active per token. The good news for local users: MoE means each token only touches ~5–6% of the weights at inference time, so smart kernels (llama.cpp's MoE-aware loader, ktransformers) can keep total memory pressure manageable. The bad news: you still need to hold the cold weights somewhere, and ~370 GB at Q4 is bigger than any consumer GPU.

The practical home options are: (1) Mac Studio M3 Ultra 192 GB at Q4 with ~6 tok/s, (2) a 4-GPU EPYC server with the experts swapped on demand from fast NVMe (~3 tok/s), or (3) hosted access via Together / Hyperbolic for everyday use with a local distill for offline work.

DeepSeek-Coder-V2, the local programming pair

DeepSeek-Coder-V2-Lite (16B MoE) is the workhorse for local coding agents, it slots into Continue.dev, Aider, and Cursor's local-mode at sensible speed on any 12 GB+ card. On Aider Polyglot we measure it within 4 points of GPT-4-Turbo on real-world refactor tasks. The full 236B variant scores higher but the Lite is enough for daily work.

Continue.dev local setup

# ~/.continue/config.yaml
models:
  - title: "DeepSeek Coder Local"
    provider: ollama
    model: deepseek-coder-v2:16b-lite-instruct-q4_K_M
    apiBase: http://localhost:11434

DeepSeek vs Llama 3, when to choose which

Choose DeepSeek when…

  • • You need explicit reasoning / chain-of-thought
  • • Coding tasks are the primary use case
  • • Math and analysis problems are common
  • • You want frontier-quality at smaller (distilled) sizes
  • • Open-source license is non-negotiable

Choose Llama 3 when…

  • • You want the broadest tooling support
  • • Multilingual chat across > 8 languages matters
  • • Vision / multimodal (Llama 3.2-Vision) needed
  • • Tight, snappy token generation matters more than depth
  • • You'll be fine-tuning, Llama 3's ecosystem is deeper

Tuning DeepSeek for your hardware

MoE flags

When running DeepSeek-V3 or Coder-V2 (full), use llama.cpp's --n-cpu-moe or the ktransformers backend so inactive experts stay in CPU RAM. This lets a 24 GB GPU host parts of a 236B MoE.

Context budget

R1 distills generate verbose CoT. Default context of 4k is too small; bump --ctx-size 16384 and reserve ~25% extra VRAM. KV cache at fp16 is 2 bytes × layer count × hidden_dim × ctx.

System prompt

R1 distills are sensitive to the system prompt. Keep it short and end with "Think step-by-step inside <think> tags before answering." Quality jumps measurably on math evals.

Quant choice

Prefer Q5_K_M over Q4 for R1 distills. MoE models are more quantization-sensitive than dense, the routing decisions are precise. Q4 still works; Q3 collapses badly.

Frequently asked questions

Can I run DeepSeek-R1 locally?

The full DeepSeek-R1 (671B MoE) is impractical on consumer hardware. The official distilled R1 variants (Qwen-7B / 14B / 32B / Llama-8B / 70B) run very well locally. R1-Distill-Qwen-32B Q4 is the sweet spot, fits in 24 GB, has near-R1 reasoning quality.

Best GPU for DeepSeek-Coder-V2?

DeepSeek-Coder-V2 16B (Lite) runs comfortably on an RTX 3060 12 GB. The full 236B MoE needs a workstation, Mac Studio M3 Ultra 128 GB+ or a multi-GPU workstation with 96 GB+ aggregate VRAM.

What quantization should I use for DeepSeek?

Q4_K_M is the default. DeepSeek's MoE routing is more sensitive to aggressive quantization than dense models, avoid Q2 / Q3 if you can. For reasoning models (R1 distills), prefer Q5_K_M when VRAM allows.

How fast is DeepSeek-R1-Distill-32B on a 4090?

We measured 33–38 tok/s at Q4_K_M, batch 1, 4k context on llama.cpp 0.4+ with CUDA. Drops to ~24 tok/s once the context window fills past 8k.

Can a Mac Studio run DeepSeek V3?

A Mac Studio M3 Ultra with 192 GB unified memory can host DeepSeek V3 (671B MoE) at Q4 with usable per-token latency thanks to MoE's lazy expert loading. It is not fast, expect ~5–8 tok/s, but it works.

Is DeepSeek better than Llama 3 for coding?

On Aider Polyglot and SWE-Bench evals, DeepSeek-Coder-V2 currently outranks Llama 3.1 70B by a meaningful margin on real-world code editing. For pure chat / instruction following, Llama 3 70B remains highly competitive.

Related guides

GPU

Best GPU for Local LLMs (2026)

Hands-on tested ranking of the best GPUs for running Llama 3, DeepSeek, Mixtral, and other large language models on your own hardware. VRAM, tokens/sec, $/perf, updated May 2026.

Read guide Software

Ollama Hardware Requirements, 2026 Guide

Minimum and recommended hardware for every Ollama model size from 3B to 70B+. RAM, VRAM, CPU, storage. NVIDIA vs Apple Silicon vs AMD ROCm vs Intel. Real measurements, not specs.

Read guide Benchmarks

llama.cpp Benchmarks, GPU, NPU & CPU 2026

Comprehensive llama.cpp benchmarks across NVIDIA GPUs, Apple Silicon, AMD Ryzen AI, and Intel CPUs. Q4_K_M / Q5_K_M / Q8_0 / F16 quantizations. Prompt-processing vs token-generation.

Read guide Server

Local AI Server Builds, Dev Box to Rack 2026

Four production-grade local AI server builds. Single-GPU dev box, dual-3090 home server, EPYC + 4× GPU workstation, rackmount inference cluster. Power, networking, IPMI, headless Linux.

Read guide

Stay Ahead of the AI Curve

Get weekly AI hardware news, benchmark updates, and deals in your inbox. Founding-subscriber list, be one of the first.

&check; No spam&check; Weekly digest&check; Unsubscribe anytime

Affiliate disclosure: As an Amazon Associate, MyAIHardware.com earns from qualifying purchases. DeepSeek performance numbers were measured before any vendor partnerships.