This distillation transfers R1's chain-of-thought reasoning into a 32 B dense Qwen-2.5 backbone — and it beats GPT-4o-mini and Claude 3.5 Sonnet on AIME and MATH benchmarks. At Q4_K_M it fits in 24 GB (RTX 4090 / 5090 / 3090) with usable context, making it the most accessible top-tier reasoning model for local users. Drawback: every output is preceded by a long <think> trace, so latency-to-first-useful-token is high.
DeepSeek-R1-Distill-Qwen-32B needs at minimum 19.4 GB of VRAM at Q4_K_M quantization (70.4 GB at FP16). The cheapest GPU that comfortably fits with KV-cache headroom is the Apple Mac mini (M4, 32 GB) (32 GB VRAM, $999 MSRP). Measured throughput hits 620 tok/s on Groq LPU (per chip). This model fits a single consumer GPU under 48 GB, so a one-card build works.
Weights only. Add ~20-30% for KV cache at typical context lengths.
| Quant | VRAM | Quality | Notes |
|---|---|---|---|
| FP16 | 70.4 GB | Reference | Training-precision weights |
| Q8_0 | 35.2 GB | Near-lossless | 8-bit, ~0.1% perplexity hit |
| Q6_K | 28.5 GB | Very high | 6-bit, near Q8 quality |
| Q5_K_M | 24.3 GB | High | Strong middle ground |
| Q4_K_M | 19.4 GB | Balanced (recommended) | Default for local deployments |
| Q4_0 | 19.8 GB | Legacy 4-bit | Older GGUF, kept for compatibility |
| Q3_K_M | 15.1 GB | Lossy | When VRAM is very tight |
| Q2_K | 11.3 GB | Extreme | Rescue option, quality degrades visibly |
Filtered from MyAIHardware's GPU database. FP16 needs 70.4 GB; Q4_K_M needs 19.4 GB.
36 measurement(s) for this model from MyAIHardware's benchmark database.
| Device | Speed | Quant | Context | Power |
|---|---|---|---|---|
| Groq LPU (per chip) | 620 tok/s | FP16 | 8192 | 215W |
| NVIDIA B200 192GB | 480 tok/s | Q4_K_M | 8192 | 1000W |
| NVIDIA B200 192GB | 385 tok/s | Q4_K_M | 8192 | 1000W |
| NVIDIA H200 141GB | 305 tok/s | Q4_K_M | 8192 | 700W |
| NVIDIA H100 SXM5 80GB | 285 tok/s | Q4_K_M | 16384 | 700W |
| AMD Instinct MI300X 192GB | 248 tok/s | Q4_K_M | 16384 | 750W |
| NVIDIA H100 SXM5 80GB | 245 tok/s | Q4_K_M | 8192 | 700W |
| NVIDIA H200 141GB | 232 tok/s | Q4_K_M | 8192 | 700W |
| AMD Instinct MI300X 192GB | 215 tok/s | Q4_K_M | 8192 | 750W |
| NVIDIA H100 SXM5 80GB | 198 tok/s | Q4_K_M | 8192 | 700W |
| NVIDIA GeForce RTX 5090 32GB | 185 tok/s | Q4_K_M | 8192 | 575W |
| AMD Instinct MI300X 192GB | 168 tok/s | Q4_K_M | 8192 | 750W |
| NVIDIA GeForce RTX 5090 32GB | 165 tok/s | Q4_K_M | 8192 | 575W |
| NVIDIA DGX Spark (Project DIGITS, 128GB) | 145 tok/s | Q4_K_M | 8192 | 240W |
| NVIDIA GeForce RTX 5080 16GB | 138 tok/s | Q4_K_M | 8192 | 360W |
| NVIDIA L40S 48GB | 138 tok/s | Q4_K_M | 8192 | 350W |
| NVIDIA GeForce RTX 4090 24GB | 125 tok/s | Q4_K_M | 8192 | 450W |
| NVIDIA GeForce RTX 5080 16GB | 122 tok/s | Q4_K_M | 8192 | 360W |
| NVIDIA GeForce RTX 4090 24GB | 118 tok/s | Q4_K_M | 8192 | 450W |
| NVIDIA GeForce RTX 5070 Ti 16GB | 108 tok/s | Q4_K_M | 8192 | 300W |
Official weights + popular runtime tags.
ollama pull deepseek-r1:32blms get bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUFStep-by-step guides that use this model.
Models in a similar size or capability class.
Use these in papers, blog posts, or internal docs.
@misc{deepseek-r1-distill-qwen-32b-2025,
title={DeepSeek-R1-Distill-Qwen-32B},
author={DeepSeek AI},
year={2025},
url={https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B}
}DeepSeek AI (2025). DeepSeek-R1-Distill-Qwen-32B [Model card]. Hugging Face. https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
DeepSeek-R1-Distill-Qwen-32B (DeepSeek R1, DeepSeek AI, 2025). Available at https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B.