DeepSeek-Coder-V2 16B (Lite) is a 16 B MoE with 2.4 B activated params, designed for fast code completion. It matches GPT-4 Turbo on HumanEval and supports 338 programming languages with a 128K context window. At Q4_K_M it runs at 60-100 tok/s on a single RTX 4090, making it the strongest local code assistant in this class. The bigger 236 B Coder-V2 variant exists for datacenter use.
DeepSeek-Coder-V2 16B needs at minimum 9.7 GB of VRAM at Q4_K_M quantization (35.2 GB at FP16). The cheapest GPU that comfortably fits with KV-cache headroom is the Intel Arc Pro B50 (16 GB VRAM, $299 MSRP). Community benchmark submissions are open. This model fits a single consumer GPU under 48 GB, so a one-card build works.
Weights only. Add ~20-30% for KV cache at typical context lengths.
| Quant | VRAM | Quality | Notes |
|---|---|---|---|
| FP16 | 35.2 GB | Reference | Training-precision weights |
| Q8_0 | 17.6 GB | Near-lossless | 8-bit, ~0.1% perplexity hit |
| Q6_K | 14.3 GB | Very high | 6-bit, near Q8 quality |
| Q5_K_M | 12.1 GB | High | Strong middle ground |
| Q4_K_M | 9.7 GB | Balanced (recommended) | Default for local deployments |
| Q4_0 | 9.9 GB | Legacy 4-bit | Older GGUF, kept for compatibility |
| Q3_K_M | 7.6 GB | Lossy | When VRAM is very tight |
| Q2_K | 5.6 GB | Extreme | Rescue option, quality degrades visibly |
Filtered from MyAIHardware's GPU database. FP16 needs 35.2 GB; Q4_K_M needs 9.7 GB.
We don't have community benchmarks for this exact model yet.
Official weights + popular runtime tags.
ollama pull deepseek-coder-v2lms get bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUFStep-by-step guides that use this model.
Models in a similar size or capability class.
Use these in papers, blog posts, or internal docs.
@misc{deepseek-coder-v2-16b-2024,
title={DeepSeek-Coder-V2 16B},
author={DeepSeek AI},
year={2024},
url={https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct}
}DeepSeek AI (2024). DeepSeek-Coder-V2 16B [Model card]. Hugging Face. https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct
DeepSeek-Coder-V2 16B (DeepSeek Coder, DeepSeek AI, 2024). Available at https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct.