Llama 3.2 3B punches well above its weight thanks to knowledge distillation from the 8B model. It's the sweet spot for AI PCs with NPUs (Intel Core Ultra, Snapdragon X, Ryzen AI) and runs at 50-100+ tok/s on midrange consumer GPUs. Tool-use and short-form summarization are particularly strong; it can struggle with long reasoning chains.
Llama 3.2 3B needs at minimum 1.8 GB of VRAM at Q4_K_M quantization (6.6 GB at FP16). The cheapest GPU that comfortably fits with KV-cache headroom is the Intel Arc A380 6GB (6 GB VRAM, $139 MSRP). Community benchmark submissions are open. This model fits a single consumer GPU under 48 GB, so a one-card build works.
Weights only. Add ~20-30% for KV cache at typical context lengths.
| Quant | VRAM | Quality | Notes |
|---|---|---|---|
| FP16 | 6.6 GB | Reference | Training-precision weights |
| Q8_0 | 3.3 GB | Near-lossless | 8-bit, ~0.1% perplexity hit |
| Q6_K | 2.7 GB | Very high | 6-bit, near Q8 quality |
| Q5_K_M | 2.3 GB | High | Strong middle ground |
| Q4_K_M | 1.8 GB | Balanced (recommended) | Default for local deployments |
| Q4_0 | 1.9 GB | Legacy 4-bit | Older GGUF, kept for compatibility |
| Q3_K_M | 1.4 GB | Lossy | When VRAM is very tight |
| Q2_K | 1.1 GB | Extreme | Rescue option, quality degrades visibly |
Filtered from MyAIHardware's GPU database. FP16 needs 6.6 GB; Q4_K_M needs 1.8 GB.
We don't have community benchmarks for this exact model yet.
Official weights + popular runtime tags.
ollama pull llama3.2:3blms get bartowski/Llama-3.2-3B-Instruct-GGUFStep-by-step guides that use this model.
Models in a similar size or capability class.
Use these in papers, blog posts, or internal docs.
@misc{llama-3-2-3b-2024,
title={Llama 3.2 3B},
author={Meta AI},
year={2024},
url={https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct}
}Meta AI (2024). Llama 3.2 3B [Model card]. Hugging Face. https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
Llama 3.2 3B (Llama 3.2, Meta AI, 2024). Available at https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct.