Llama 3.3 70B is Meta's late-2024 refresh that distills 405B-style instruction quality into the 70B footprint — it beats Llama 3.1 70B on MATH, GPQA, and IFEval while keeping the same VRAM and tok/s envelope. It is the default 70B-class recommendation for new local deployments. Hardware requirements are identical to Llama 3.1 70B (24 GB at Q4 with reduced context).
Llama 3.3 70B needs at minimum 42.4 GB of VRAM at Q4_K_M quantization (154 GB at FP16). The cheapest GPU that comfortably fits with KV-cache headroom is the Apple Mac mini (M4 Pro, 64 GB) (64 GB VRAM, $2,199 MSRP). Measured throughput hits 540 tok/s on NVIDIA H200 141GB. This model fits a single consumer GPU under 48 GB, so a one-card build works.
Weights only. Add ~20-30% for KV cache at typical context lengths.
| Quant | VRAM | Quality | Notes |
|---|---|---|---|
| FP16 | 154.0 GB | Reference | Training-precision weights |
| Q8_0 | 77.0 GB | Near-lossless | 8-bit, ~0.1% perplexity hit |
| Q6_K | 62.4 GB | Very high | 6-bit, near Q8 quality |
| Q5_K_M | 53.1 GB | High | Strong middle ground |
| Q4_K_M | 42.4 GB | Balanced (recommended) | Default for local deployments |
| Q4_0 | 43.3 GB | Legacy 4-bit | Older GGUF, kept for compatibility |
| Q3_K_M | 33.1 GB | Lossy | When VRAM is very tight |
| Q2_K | 24.6 GB | Extreme | Rescue option, quality degrades visibly |
Filtered from MyAIHardware's GPU database. FP16 needs 154.0 GB; Q4_K_M needs 42.4 GB.
52 measurement(s) for this model from MyAIHardware's benchmark database.
| Device | Speed | Quant | Context | Power |
|---|---|---|---|---|
| NVIDIA H200 141GB | 540 tok/s | Q4_K_M | 4096 | 700W |
| Cerebras WSE-3 (CS-3) | 450 tok/s | FP16 | 8192 | 23000W |
| 8× NVIDIA H100 SXM5 80GB | 412 tok/s | Q4_K_M | 8192 | 5600W |
| NVIDIA H100 SXM5 80GB | 410 tok/s | Q4_K_M | 4096 | 700W |
| AMD Instinct MI300X 192GB | 380 tok/s | Q4_K_M | 4096 | 750W |
| Groq LPU (8-chip rack) | 285 tok/s | FP16 | 8192 | 1720W |
| NVIDIA B200 192GB | 138 tok/s | Q4_K_M | 8192 | 1000W |
| Google TPU v5p | 110 tok/s | INT8 | 8192 | 700W |
| NVIDIA B200 192GB | 98 tok/s | Q8_0 | 8192 | 1000W |
| NVIDIA GH200 Grace Hopper 480GB | 95 tok/s | Q4_K_M | 8192 | 1000W |
| AWS Trainium2 | 92 tok/s | INT8 | 8192 | 500W |
| NVIDIA H200 141GB | 84 tok/s | Q4_K_M | 8192 | 700W |
| NVIDIA GeForce RTX 5090 32GB | 78 tok/s | Q4_K_M | 4096 | 575W |
| AMD Instinct MI300X 192GB | 72 tok/s | Q4_K_M | 8192 | 750W |
| NVIDIA H100 SXM5 80GB | 66 tok/s | Q4_K_M | 4096 | 700W |
| Google TPU v5e | 62 tok/s | INT8 | 4096 | 170W |
| NVIDIA H200 141GB | 58 tok/s | Q8_0 | 8192 | 700W |
| AWS Trainium2 | 58 tok/s | Q4_K_M | 8192 | 500W |
| AMD Instinct MI250X 128GB | 52 tok/s | Q4_K_M | 8192 | 560W |
| AMD Instinct MI300X 192GB | 48 tok/s | Q8_0 | 8192 | 750W |
Official weights + popular runtime tags.
ollama pull llama3.3lms get bartowski/Llama-3.3-70B-Instruct-GGUFStep-by-step guides that use this model.
Models in a similar size or capability class.
Use these in papers, blog posts, or internal docs.
@misc{llama-3-3-70b-2024,
title={Llama 3.3 70B},
author={Meta AI},
year={2024},
url={https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct}
}Meta AI (2024). Llama 3.3 70B [Model card]. Hugging Face. https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct
Llama 3.3 70B (Llama 3.3, Meta AI, 2024). Available at https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct.