VRAM
192 GB
TDP
1000 W
MSRP
$40k
Perf/W
0.14 emb/s/W
Cost/1K tok
$0.0031/k
Tested
2024-12-26
NVIDIA B200 192GB produces approximately 138.0 tok/s on Llama 3 70B at Q4_K_M quantization (batch 1, 8192-token context, 192GB VRAM, 1000W TDP). That figure comes from 4 measured runs on TensorRT-LLM 0.15. The 70B model needs roughly 40GB of VRAM at Q4, so headroom and KV cache budget matter as much as raw throughput.
NVIDIA B200 192GB — Blackwell-architecture datacenter GPU. 192GB HBM3e at 8 TB/s, 1000W TDP. Second-gen Transformer Engine with FP4 support. 2.5x the AI compute of H100. NVIDIA's 2025 flagship datacenter GPU.
192GB VRAM at 8 TB/s bandwidth makes this the fastest single-GPU inference platform available. Hosts 405B at FP8 with massive context headroom. FP4 doubles effective throughput when engines support it. Currently the ceiling for single-GPU local AI — everything above requires tensor parallelism. Priced for cloud/enterprise, not homelab.
NVIDIA B200 192GB with 192GB VRAM at 1000W TDP, scored across 12 workloads with 14 benchmark records.
Best workload
Embedding throughput
23500 emb/s
Quantization
FP16
1K context · batch 1
| Workload | Score | Quant | Context | σ | Status | Tested |
|---|---|---|---|---|---|---|
| Llama 3 70B Q4 llm | 138.0tok/s | Q4_K_M | 8K | ±12.4 | Vendor Claim | 2024-12-26 |
| Llama 3 70B Q8 llm | 98.0tok/s | Q8_0 | 8K | , | Curated Aggregate | 2024-12-30 |
| Llama 3 8B FP16 llm | 520.0tok/s | FP16 | 8K | , | Curated Aggregate | 2026-02-04 |
| SDXL image gen image | 52.0img/min | FP16 | , | , | Curated Aggregate | 2025-04-15 |
| Qwen 2.5 14B llm | 285.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2026-01-15 |
| DeepSeek-R1 7B llm | 480.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2026-02-04 |
| Whisper transcription audio | 210.0x RT | FP16 | , | , | Curated Aggregate | 2026-02-22 |
| Mistral 7B llm | 415.0tok/s | Q4_K_M | 4K | , | Curated Aggregate | 2025-01-08 |
| Gemma 2 9B llm | 332.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2025-02-12 |
| Llama 3 8B Q4 llm | 360.0tok/s | Q4_K_M | 128K | , | Curated Aggregate | 2025-03-20 |
| Embedding throughput embedding | 23500.0emb/s | FP16 | 1K | , | Curated Aggregate | 2025-01-18 |
| Asure 12B llm | 350.0tok/s | Q4_K_M | 8K | ±10.5 | Vendor Claim | 2025-05-01 |
NVIDIA B200 192GB clears a 9.5/10 based on workload-normalized throughput, memory headroom, efficiency, value, trust, and coverage.
What models fit this 192GB card at different quantization levels.
Trust score
6/10
MyAI rating
9.5
Runs
1
Freshness
Stale
Source-linked row with explicit verification status.
Tested on 2025-01-18; 586 days old.
Enterprise buying note
Retailer we'd check first
Amazon search plus reseller quotes
For datacenter and accelerator parts, Amazon is useful for spotting live listings, accessories, or used pulls, but serious procurement usually happens through integrators, brokers, or cloud partners.
Enterprise procurement, not retail
This silicon is typically acquired through an authorized OEM partner, system integrator, or hyperscaler reseller. For on-demand access, compare hourly rates at RunPod, Vast.ai, Lambda Labs, or your existing cloud provider before committing to capital expenditure.
This is one of the highest-scoring parts in our database, so it is worth tracking when inventory lands near fair-market pricing.
Affiliate note: we do not have a device-level ASIN yet, so this opens tagged Amazon search results for the exact product name.