Mixtral 8x22B is the bigger sibling — 141 B total, 39 B active, with stronger multilingual and reasoning performance. VRAM is brutal: ~310 GB FP16, ~85 GB Q4_K_M. Realistically a 4x RTX 6000 Ada (192 GB) or 2x H100 (160 GB at Q4) setup. Beaten on raw quality by Llama 3.1 70B at a smaller footprint, so adoption has been limited.
Mixtral 8x22B needs at minimum 85.3 GB of VRAM at Q4_K_M quantization (310.2 GB at FP16). The cheapest GPU that comfortably fits with KV-cache headroom is the Intel Data Center GPU Max 1550 (128 GB VRAM, $2,500 MSRP). Community benchmark submissions are open. This model exceeds 48 GB at Q4, so plan for a 2-or-more-GPU split.
Weights only. Add ~20-30% for KV cache at typical context lengths.
| Quant | VRAM | Quality | Notes |
|---|---|---|---|
| FP16 | 310.2 GB | Reference | Training-precision weights |
| Q8_0 | 155.1 GB | Near-lossless | 8-bit, ~0.1% perplexity hit |
| Q6_K | 125.6 GB | Very high | 6-bit, near Q8 quality |
| Q5_K_M | 107.0 GB | High | Strong middle ground |
| Q4_K_M | 85.3 GB | Balanced (recommended) | Default for local deployments |
| Q4_0 | 87.2 GB | Legacy 4-bit | Older GGUF, kept for compatibility |
| Q3_K_M | 66.7 GB | Lossy | When VRAM is very tight |
| Q2_K | 49.6 GB | Extreme | Rescue option, quality degrades visibly |
Filtered from MyAIHardware's GPU database. FP16 needs 310.2 GB; Q4_K_M needs 85.3 GB.
| GPU | Per-card VRAM | 2x | 4x | 8x |
|---|---|---|---|---|
| NVIDIA H100 SXM5 | 80 GB | 160 | 320 | 640 |
| NVIDIA A100 80GB | 80 GB | 160 | 320 | 640 |
| Apple Mac mini (M4 Pro, 64 GB) | 64 GB | 128 | 256 | 512 |
| Apple Mac Studio (M1 Max, 64 GB) | 64 GB | 128 | 256 | 512 |
| Apple Mac Studio (M1 Ultra, 64 GB) | 64 GB | 128 | 256 | 512 |
| Apple Mac Studio (M2 Max, 64 GB) | 64 GB | 128 | 256 | 512 |
We don't have community benchmarks for this exact model yet.
Official weights + popular runtime tags.
ollama pull mixtral:8x22blms get MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUFStep-by-step guides that use this model.
Models in a similar size or capability class.
Use these in papers, blog posts, or internal docs.
@misc{mixtral-8x22b-2024,
title={Mixtral 8x22B},
author={Mistral AI},
year={2024},
url={https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1}
}Mistral AI (2024). Mixtral 8x22B [Model card]. Hugging Face. https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1
Mixtral 8x22B (Mistral, Mistral AI, 2024). Available at https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1.