Open Model Catalog
30 deep-dive pages for the highest-traffic open models. Each page has exact VRAM tables, GPU-fit recommendations, community benchmarks, and download commands.
Nomic Embed Text v1.5
Nomic Embed Text v1.5 is the leading open long-context embedding model — supports 8192 tokens (16x BGE-Large) and uses Matryoshka representation learning, so you can truncate the 768-dim output to 256 or 128 dims with minimal accuracy loss. Apache 2.0 licensed and trained on fully open data. Tiny 137 M footprint runs on CPU; on GPU it embeds 200k+ chunks/sec.
BGE-Large EN v1.5
BGE-Large EN v1.5 from BAAI is the reference English embedding model for RAG pipelines — produces 1024-dim vectors, scores near the top of MTEB retrieval benchmarks. The tiny 335 M footprint runs on CPU comfortably or 100k+ embeddings/sec on a single RTX 4090. The 512-token context cap is the main limitation versus Nomic Embed v1.5.
Llama 3.2 1B
Llama 3.2 1B is the smallest official Llama, designed for on-device deployment — it runs comfortably on a Snapdragon X Elite NPU, an Apple Neural Engine, or even a Raspberry Pi 5. Quality is naturally limited; it's best for summarization, classification, and tool-routing tasks rather than long-form generation. Expect 30-50 tok/s on midrange phones.
Whisper Large v3
Whisper Large v3 is OpenAI's flagship open speech-to-text model — supports 99 languages with leading WER scores. The Apache 2.0 license and tiny ~3 GB FP16 footprint make it deployable everywhere from Raspberry Pi (via whisper.cpp Q4) to datacenter. On an RTX 4090 it transcribes at ~50x real-time. Known weakness: occasional hallucinations on silence or low-resource languages.
WhisperX
WhisperX wraps Whisper Large v3 with faster-whisper batched inference, wav2vec2 forced alignment for word-level timestamps, and pyannote diarization for speaker identification — achieving up to 70x real-time on an RTX 4090. It's the recommended pipeline for subtitle generation, podcast transcription, and meeting recording analysis. Trade-off: more dependencies and a slightly more complex install than vanilla whisper.cpp.
Llama 3.2 3B
Llama 3.2 3B punches well above its weight thanks to knowledge distillation from the 8B model. It's the sweet spot for AI PCs with NPUs (Intel Core Ultra, Snapdragon X, Ryzen AI) and runs at 50-100+ tok/s on midrange consumer GPUs. Tool-use and short-form summarization are particularly strong; it can struggle with long reasoning chains.
SDXL 1.0
SDXL 1.0 is the de-facto open image-generation model — its 3.5 B parameter U-Net plus dual text encoders produces 1024×1024 images with strong composition. The ecosystem (ControlNet, LoRAs, IP-Adapter, refiner) is unmatched among open models. Runs on any 8 GB+ GPU; an RTX 4090 hits ~28 img/min at 30 steps. FLUX.1 dev has surpassed it on quality but the SDXL tooling lead remains significant.
Phi-3.5 Mini 3.8B
Phi-3.5 Mini is Microsoft's curated-synthetic-data 3.8 B model — rivals 7-13 B models on reasoning despite the small size. Particularly notable for the 128K context window in such a small footprint. Runs at 100+ tok/s on midrange consumer GPUs at Q4_K_M (only ~2.3 GB VRAM). Weakness: limited world knowledge versus naturally-trained peers.
Phi-4 Mini 3.8B
Phi-4 Mini is Microsoft's February 2025 successor to Phi-3.5 Mini — same 3.8 B footprint but with first-class function calling, a 128K context window, and a tokenizer overhaul that doubles non-English efficiency. It beats Phi-3.5 Mini on every reasoning benchmark and approaches Phi-4 14B on math while running at ~2.3 GB VRAM at Q4_K_M. MIT licensed. The realistic deployment target: AI PCs with NPUs, mini PCs, edge servers, and even a Raspberry Pi 5 via llama.cpp. For a Lagos solo dev on a 6 GB laptop GPU, this is the local agent model that actually fits.
Qwen 2.5 7B
Qwen 2.5 7B is Alibaba's flagship small model — particularly strong on Chinese, math, and code benchmarks where it edges out Llama 3.1 8B. The Apache 2.0 license makes it the preferred 7B for commercial deployments. Runs at 60-100 tok/s on a single RTX 4060 8 GB at Q4_K_M.
Mistral 7B
Mistral 7B v0.3 remains the reference small open model — first to popularize sliding-window attention and grouped-query attention. It's been surpassed on raw benchmarks by Llama 3.1 8B and Qwen 2.5 7B, but its Apache 2.0 license and massive ecosystem of fine-tunes keep it relevant. Excellent base for custom fine-tunes.
Llama 3.1 8B
Llama 3.1 8B is Meta's best-in-class small dense model — competitive with much larger 13B-class models on reasoning and instruction following. It's fast enough for single-GPU local chat on 8 GB cards at Q4 and ships with full 128K context support. The weakness is that it still trails 70B-class models on multi-step reasoning and tool use.
Granite 3.2 8B Instruct
Granite 3.2 8B Instruct is IBM's February 2025 reasoning model — same 8 B footprint as Llama 3.1 8B but trained on a fully audited corpus that IBM will indemnify enterprise customers against. It ships with a toggleable thinking mode for chain-of-thought reasoning and matches or beats Llama 3.1 8B on most leaderboards. The pitch is not raw quality, it is the legal and procurement story: Apache 2.0 weights, documented training data, and IBM's enterprise SLA on watsonx.ai for buyers who cannot deploy Meta or Alibaba weights. Runs at 60-100 tok/s on a single RTX 4060 8 GB at Q4_K_M. For a regulated bank in Istanbul or a public-sector buyer in Lagos, this is often the only 8B model that clears procurement.
Llama 3.2 Vision 11B
Llama 3.2 Vision 11B is Meta's accessible open multimodal model — handles image-grounded QA, chart reading, and document understanding well. Fits in 16 GB VRAM at Q4_K_M. Competitive with GPT-4V on common tasks but lags on dense-text OCR and very detailed visual reasoning where the 90B variant pulls ahead.
FLUX.1 [dev]
FLUX.1 [dev] from Black Forest Labs (ex-Stability researchers) is the highest-quality open image model — beats SDXL and approaches Midjourney v6 on photorealism and prompt following. The 12 B parameter footprint demands ~24 GB VRAM at FP16, ~7 GB at Q4 (NF4) using community quants. The non-commercial license is the main adoption blocker; FLUX.1 [schnell] is the Apache-licensed alternative.
Qwen 2.5 14B
Qwen 2.5 14B sits in the 'serious assistant' sweet spot — clearly better than 7B on multi-step reasoning while still fitting comfortably in 16 GB VRAM at Q4_K_M. It's a good choice for a single RTX 4070 Ti Super or 4080 build. Trade-off vs 7B is roughly 2x slower tok/s.
Phi-4 14B
Phi-4 14B is Microsoft's late-2024 release that pushes synthetic-data training to its limit — beats Llama 3.3 70B on MATH and GPQA Diamond despite being 5x smaller. The MIT license is permissive for commercial use. Fits in 16 GB VRAM at Q4_K_M with usable context. Weakness: only a 16K context window, and limited multilingual depth versus Qwen.
DeepSeek-Coder-V2 16B
DeepSeek-Coder-V2 16B (Lite) is a 16 B MoE with 2.4 B activated params, designed for fast code completion. It matches GPT-4 Turbo on HumanEval and supports 338 programming languages with a 128K context window. At Q4_K_M it runs at 60-100 tok/s on a single RTX 4090, making it the strongest local code assistant in this class. The bigger 236 B Coder-V2 variant exists for datacenter use.
Llama 4 Maverick 17B
Llama 4 Maverick is Meta's flagship multimodal MoE — 400 B total parameters across 128 experts with 17 B activated per token. Benchmarks place it near GPT-4o on reasoning and beating Llama 3.1 405B at far lower per-token cost. Realistically it requires a 4x or 8x H100 / B200 cluster (~900 GB at FP16, ~245 GB at Q4_K_M). Not a homelab model.
DeepSeek-R1-Distill-Qwen-32B
This distillation transfers R1's chain-of-thought reasoning into a 32 B dense Qwen-2.5 backbone — and it beats GPT-4o-mini and Claude 3.5 Sonnet on AIME and MATH benchmarks. At Q4_K_M it fits in 24 GB (RTX 4090 / 5090 / 3090) with usable context, making it the most accessible top-tier reasoning model for local users. Drawback: every output is preceded by a long <think> trace, so latency-to-first-useful-token is high.
Qwen 2.5 32B
Qwen 2.5 32B is the strongest model that reliably fits in a single 24 GB consumer GPU at Q4_K_M. It outperforms Llama 3.1 70B on Chinese benchmarks and is competitive on English reasoning. Combined with the Apache 2.0 license, it's the go-to model for prosumer workstation deployments.
Qwen 2.5 Coder 32B
Qwen 2.5 Coder 32B is currently the best open-weight coding model — it tops HumanEval, MBPP, and LiveCodeBench while supporting 92 programming languages and a 128K context for whole-repo reasoning. Fits in 24 GB at Q4_K_M, making it deployable on a single RTX 4090 / 5090 / 3090. The natural replacement for DeepSeek-Coder-V2 when you want dense-model quality.
Qwen 3 32B
Qwen 3 32B is the largest Qwen 3 that fits in a single 24 GB consumer GPU at Q4_K_M — and on reasoning benchmarks it closes most of the gap to the 72B sibling thanks to the hybrid thinking mode. For a Lagos or Istanbul builder running a single RTX 4090 or 5090, this is the strongest local model you can run without resorting to MoE offload. Apache 2.0, 128K context, full multilingual support. The thinking mode produces noticeably longer outputs than Qwen 2.5 32B, so expect tok/s to feel slower at identical hardware unless you disable it.
Mixtral 8x7B
Mixtral 8x7B was the first widely successful open MoE — 47 B total weights but only 12.9 B activated per token. It needs all weights in VRAM (~26 GB at Q4_K_M), so practically a single 32 GB GPU (RTX 5090, A6000) or 2x 16 GB cards. The Apache 2.0 license and competitive Llama 2 70B-class quality made it a homelab favorite, though Llama 3.1 70B has largely superseded it.
Llama 3.1 70B
Llama 3.1 70B is the workhorse of self-hosted AI in 2024-2026 — competitive with GPT-4-class proprietary models on most benchmarks. At Q4_K_M it fits in a single RTX 4090 or 5090 (24-32 GB) with reduced context, or comfortably on dual 24 GB cards. The trade-off versus 8B is roughly 5-8x slower tok/s on identical hardware.
Llama 3.3 70B
Llama 3.3 70B is Meta's late-2024 refresh that distills 405B-style instruction quality into the 70B footprint — it beats Llama 3.1 70B on MATH, GPQA, and IFEval while keeping the same VRAM and tok/s envelope. It is the default 70B-class recommendation for new local deployments. Hardware requirements are identical to Llama 3.1 70B (24 GB at Q4 with reduced context).
Qwen 2.5 72B
Qwen 2.5 72B is Alibaba's flagship dense model, neck-and-neck with Llama 3.1 70B and pulling ahead on Chinese, math, and coding. VRAM footprint mirrors Llama 70B (~158 GB FP16, ~44 GB Q4_K_M). Note the license is Qwen-specific, not Apache, which limits some commercial uses above 100M MAU.
Qwen 3 72B
Qwen 3 72B is Alibaba's April 2025 flagship dense release and the first major open model to ship a hybrid thinking mode — one set of weights, two behaviors. Set enable_thinking=true for o1-style chain-of-thought on math and code, or false for fast direct answers. It tops Qwen 2.5 72B on AIME, LiveCodeBench, and BFCL while inheriting Qwen's strong multilingual coverage (119 languages). Apache 2.0 license, identical VRAM footprint to Qwen 2.5 72B. At Q4_K_M it fits across 2x RTX 4090 (48 GB) or 1x H100 80 GB. The reasoning mode roughly doubles output tokens, so plan KV-cache headroom accordingly. See [Q4_K_M quantization](/glossary/q4-k-m) for the math.
Llama 4 Scout 17B
Llama 4 Scout is Meta's first MoE Llama — 109 B total parameters, 17 B per-token compute, 16 experts. The headline feature is a 10M-token context window (per Meta's April 2025 announcement), enabling whole-codebase or whole-book reasoning. Hardware-wise it's awkward: you must hold all 109 B weights in VRAM (~240 GB at FP16, ~66 GB at Q4_K_M from the table below), but per-token compute is only 17 B — so a single H100 80 GB at Q4 is the practical entry point for inference, or 2x H100 for FP16. See our [Q4_K_M quantization](/glossary/q4-k-m) and [MoE architecture](/glossary/moe) glossary entries for the math, and the [/benchmarks](/benchmarks) page for measured tok/s.
Mistral Large 2 123B
Mistral Large 2 is Mistral's flagship dense model — particularly strong on European languages (French, German, Spanish, Italian) and code (HumanEval ~92%). VRAM footprint is steep: ~270 GB FP16, ~75 GB Q4_K_M. Requires 2x H100, 4x A100 40 GB, or 4x RTX 6000 Ada. The Mistral Research License limits commercial deployment without a paid agreement.
Mixtral 8x22B
Mixtral 8x22B is the bigger sibling — 141 B total, 39 B active, with stronger multilingual and reasoning performance. VRAM is brutal: ~310 GB FP16, ~85 GB Q4_K_M. Realistically a 4x RTX 6000 Ada (192 GB) or 2x H100 (160 GB at Q4) setup. Beaten on raw quality by Llama 3.1 70B at a smaller footprint, so adoption has been limited.
Llama 3.1 405B
Llama 3.1 405B is the largest open-weight Llama and arguably the highest-quality open base model ever released. At FP16 it requires ~890 GB of VRAM, meaning 8x H100 / B200 minimum; at Q4_K_M it still demands ~245 GB. It's not a homelab model — but quantized aggressively across 4x A100 80 GB it becomes usable for research workloads.
DeepSeek-R1 671B
DeepSeek-R1 is the open MoE reasoning model that closed the gap with OpenAI o1 — and the first model to publish RL-from-verifiable-reward methodology. With 671 B total / 37 B active parameters, it requires datacenter hardware: ~1340 GB at FP16, ~370 GB at Q4_K_M. Practically, an 8x H100 or 8x B200 server is the floor; some users run Q4 across 8x RTX 6000 Ada (384 GB total) but throughput is constrained.
DeepSeek V3 671B
DeepSeek V3 is the 671 B MoE base model (37 B active) that DeepSeek used as the starting point for R1. It beats Llama 3.1 405B on most benchmarks at a fraction of the compute, thanks to MLA attention and FP8 mixed-precision training. Inference VRAM mirrors R1 — ~370 GB at Q4_K_M, requiring multi-GPU datacenter hardware. Recommended over R1 when you want fast responses without the <think> overhead.
DeepSeek V3-0324
DeepSeek V3-0324 is the March 24, 2025 refresh — same 685 B / 37 B-active MoE architecture as base V3, retrained with R1-style reasoning distillation and an MIT license (a meaningful upgrade from the original V3's bespoke DeepSeek license). On MATH-500, AIME, GPQA, and LiveCodeBench it lands between base V3 and R1 while still producing direct answers (no <think> trace). VRAM math is the same as base V3: ~1370 GB FP16, ~378 GB Q4_K_M — so 8x H100 or 8x B200 is the floor. For Lagos and Istanbul builders this is a cloud-API model, not a homelab one. See our [MoE architecture](/glossary/moe) glossary entry for why only 37 B of the 685 B is touched per token.