Qwen 2.5 Coder 32B is currently the best open-weight coding model — it tops HumanEval, MBPP, and LiveCodeBench while supporting 92 programming languages and a 128K context for whole-repo reasoning. Fits in 24 GB at Q4_K_M, making it deployable on a single RTX 4090 / 5090 / 3090. The natural replacement for DeepSeek-Coder-V2 when you want dense-model quality.
Qwen 2.5 Coder 32B needs at minimum 19.4 GB of VRAM at Q4_K_M quantization (70.4 GB at FP16). The cheapest GPU that comfortably fits with KV-cache headroom is the Apple Mac mini (M4, 32 GB) (32 GB VRAM, $999 MSRP). Community benchmark submissions are open. This model fits a single consumer GPU under 48 GB, so a one-card build works.
Weights only. Add ~20-30% for KV cache at typical context lengths.
| Quant | VRAM | Quality | Notes |
|---|---|---|---|
| FP16 | 70.4 GB | Reference | Training-precision weights |
| Q8_0 | 35.2 GB | Near-lossless | 8-bit, ~0.1% perplexity hit |
| Q6_K | 28.5 GB | Very high | 6-bit, near Q8 quality |
| Q5_K_M | 24.3 GB | High | Strong middle ground |
| Q4_K_M | 19.4 GB | Balanced (recommended) | Default for local deployments |
| Q4_0 | 19.8 GB | Legacy 4-bit | Older GGUF, kept for compatibility |
| Q3_K_M | 15.1 GB | Lossy | When VRAM is very tight |
| Q2_K | 11.3 GB | Extreme | Rescue option, quality degrades visibly |
Filtered from MyAIHardware's GPU database. FP16 needs 70.4 GB; Q4_K_M needs 19.4 GB.
We don't have community benchmarks for this exact model yet.
Official weights + popular runtime tags.
ollama pull qwen2.5-coder:32blms get Qwen/Qwen2.5-Coder-32B-Instruct-GGUFStep-by-step guides that use this model.
Models in a similar size or capability class.
Use these in papers, blog posts, or internal docs.
@misc{qwen-2-5-coder-32b-2024,
title={Qwen 2.5 Coder 32B},
author={Alibaba Cloud Qwen Team},
year={2024},
url={https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct}
}Alibaba Cloud Qwen Team (2024). Qwen 2.5 Coder 32B [Model card]. Hugging Face. https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct
Qwen 2.5 Coder 32B (Qwen 2.5, Alibaba Cloud Qwen Team, 2024). Available at https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct.