Qwen 3 32B is the largest Qwen 3 that fits in a single 24 GB consumer GPU at Q4_K_M — and on reasoning benchmarks it closes most of the gap to the 72B sibling thanks to the hybrid thinking mode. For a Lagos or Istanbul builder running a single RTX 4090 or 5090, this is the strongest local model you can run without resorting to MoE offload. Apache 2.0, 128K context, full multilingual support. The thinking mode produces noticeably longer outputs than Qwen 2.5 32B, so expect tok/s to feel slower at identical hardware unless you disable it.
Qwen 3 32B needs at minimum 19.4 GB of VRAM at Q4_K_M quantization (70.4 GB at FP16). The cheapest GPU that comfortably fits with KV-cache headroom is the Apple Mac mini (M4, 32 GB) (32 GB VRAM, $999 MSRP). Community benchmark submissions are open. This model fits a single consumer GPU under 48 GB, so a one-card build works.
Weights only. Add ~20-30% for KV cache at typical context lengths.
| Quant | VRAM | Quality | Notes |
|---|---|---|---|
| FP16 | 70.4 GB | Reference | Training-precision weights |
| Q8_0 | 35.2 GB | Near-lossless | 8-bit, ~0.1% perplexity hit |
| Q6_K | 28.5 GB | Very high | 6-bit, near Q8 quality |
| Q5_K_M | 24.3 GB | High | Strong middle ground |
| Q4_K_M | 19.4 GB | Balanced (recommended) | Default for local deployments |
| Q4_0 | 19.8 GB | Legacy 4-bit | Older GGUF, kept for compatibility |
| Q3_K_M | 15.1 GB | Lossy | When VRAM is very tight |
| Q2_K | 11.3 GB | Extreme | Rescue option, quality degrades visibly |
Filtered from MyAIHardware's GPU database. FP16 needs 70.4 GB; Q4_K_M needs 19.4 GB.
We don't have community benchmarks for this exact model yet.
Official weights + popular runtime tags.
ollama pull qwen3:32blms get Qwen/Qwen3-32B-GGUFStep-by-step guides that use this model.
Models in a similar size or capability class.
Use these in papers, blog posts, or internal docs.
@misc{qwen-3-32b-2025,
title={Qwen 3 32B},
author={Alibaba Cloud Qwen Team},
year={2025},
url={https://huggingface.co/Qwen/Qwen3-32B}
}Alibaba Cloud Qwen Team (2025). Qwen 3 32B [Model card]. Hugging Face. https://huggingface.co/Qwen/Qwen3-32B
Qwen 3 32B (Qwen 3, Alibaba Cloud Qwen Team, 2025). Available at https://huggingface.co/Qwen/Qwen3-32B.