Granite 3.2 8B Instruct is IBM's February 2025 reasoning model — same 8 B footprint as Llama 3.1 8B but trained on a fully audited corpus that IBM will indemnify enterprise customers against. It ships with a toggleable thinking mode for chain-of-thought reasoning and matches or beats Llama 3.1 8B on most leaderboards. The pitch is not raw quality, it is the legal and procurement story: Apache 2.0 weights, documented training data, and IBM's enterprise SLA on watsonx.ai for buyers who cannot deploy Meta or Alibaba weights. Runs at 60-100 tok/s on a single RTX 4060 8 GB at Q4_K_M. For a regulated bank in Istanbul or a public-sector buyer in Lagos, this is often the only 8B model that clears procurement.
Granite 3.2 8B Instruct needs at minimum 4.8 GB of VRAM at Q4_K_M quantization (17.6 GB at FP16). The cheapest GPU that comfortably fits with KV-cache headroom is the Intel Arc A580 8GB (8 GB VRAM, $179 MSRP). Community benchmark submissions are open. This model fits a single consumer GPU under 48 GB, so a one-card build works.
Weights only. Add ~20-30% for KV cache at typical context lengths.
| Quant | VRAM | Quality | Notes |
|---|---|---|---|
| FP16 | 17.6 GB | Reference | Training-precision weights |
| Q8_0 | 8.8 GB | Near-lossless | 8-bit, ~0.1% perplexity hit |
| Q6_K | 7.1 GB | Very high | 6-bit, near Q8 quality |
| Q5_K_M | 6.1 GB | High | Strong middle ground |
| Q4_K_M | 4.8 GB | Balanced (recommended) | Default for local deployments |
| Q4_0 | 5.0 GB | Legacy 4-bit | Older GGUF, kept for compatibility |
| Q3_K_M | 3.8 GB | Lossy | When VRAM is very tight |
| Q2_K | 2.8 GB | Extreme | Rescue option, quality degrades visibly |
Filtered from MyAIHardware's GPU database. FP16 needs 17.6 GB; Q4_K_M needs 4.8 GB.
We don't have community benchmarks for this exact model yet.
Official weights + popular runtime tags.
ollama pull granite3.2lms get ibm-granite/granite-3.2-8b-instruct-GGUFStep-by-step guides that use this model.
Models in a similar size or capability class.
Use these in papers, blog posts, or internal docs.
@misc{granite-3-2-8b-2025,
title={Granite 3.2 8B Instruct},
author={Granite},
year={2025},
url={https://huggingface.co/ibm-granite/granite-3.2-8b-instruct}
}Granite (2025). Granite 3.2 8B Instruct [Model card]. Hugging Face. https://huggingface.co/ibm-granite/granite-3.2-8b-instruct
Granite 3.2 8B Instruct (Granite, Granite, 2025). Available at https://huggingface.co/ibm-granite/granite-3.2-8b-instruct.