VRAM
141 GB
TDP
700 W
MSRP
$30k
Perf/W
0.12 emb/s/W
Cost/1K tok
$0.0038/k
Tested
2024-04-06
NVIDIA H200 141GB produces approximately 84.0 tok/s on Llama 3 70B at Q4_K_M quantization (batch 1, 8192-token context, 141GB VRAM, 700W TDP). That figure comes from 5 measured runs on TensorRT-LLM 0.14. The 70B model needs roughly 40GB of VRAM at Q4, so headroom and KV cache budget matter as much as raw throughput.
NVIDIA H200 141GB — Hopper-generation capacity-upgrade GPU. 141GB HBM3e at 4.8 TB/s, 700W TDP. Same compute dies as H100 with dramatically more and faster memory. Purpose-built for large-model inference.
141GB VRAM hosts Llama 3.1 405B at FP8 comfortably. The go-to for frontier-model serving in 2025-2026. ~2x the memory bandwidth of H100 means ~1.8x tok/s on memory-bound decode. The capacity play — not a compute upgrade over H100, but the VRAM ceiling boost is what matters for the largest open models.
The capacity king. Right answer when 80GB isn't enough and you need 141GB. For everything else, H100 or L40S are more economical.
405B-class models at FP8 with context headroom — the capacity play for frontier open-weight models
HBM3e thermal management at sustained 4.8 TB/s bandwidth — requires proper datacenter airflow. Memory errors under thermal stress more common than H100 due to higher density.
Ubuntu 22.04/24.04 LTS (reference), RHEL 9
700W SXM. Datacenter infrastructure required. The 141GB HBM3e draws more board power than H100 under sustained memory load.
NVIDIA H200 141GB with 141GB VRAM at 700W TDP, scored across 11 workloads with 14 benchmark records.
Best workload
Embedding throughput
15200 emb/s
Quantization
FP16
1K context · batch 1
| Workload | Score | Quant | Context | σ | Status | Tested |
|---|---|---|---|---|---|---|
| Llama 3 70B Q4 llm | 540.0tok/s | Q4_K_M | 4K | , | Curated Aggregate | 2024-11-22 |
| Llama 3 70B Q8 llm | 58.0tok/s | Q8_0 | 8K | , | Curated Aggregate | 2024-04-10 |
| Llama 3 8B FP16 llm | 340.0tok/s | FP16 | 4K | , | Curated Aggregate | 2025-11-18 |
| Mistral 7B llm | 305.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2026-01-28 |
| SDXL image gen image | 36.0img/min | FP16 | , | , | Curated Aggregate | 2026-02-15 |
| DeepSeek-R1 7B llm | 305.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2026-04-20 |
| Gemma 2 9B llm | 220.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2024-07-30 |
| Qwen 2.5 14B llm | 178.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2024-12-15 |
| Llama 3 8B Q4 llm | 215.0tok/s | Q4_K_M | 128K | , | Curated Aggregate | 2025-02-04 |
| Embedding throughput embedding | 15200.0emb/s | FP16 | 1K | , | Curated Aggregate | 2024-09-08 |
| Asure 12B llm | 230.0tok/s | Q4_K_M | 8K | ±9.2 | Vendor Claim | 2025-05-01 |
NVIDIA H200 141GB clears a 9.4/10 based on workload-normalized throughput, memory headroom, efficiency, value, trust, and coverage.
What models fit this 141GB card at different quantization levels.
Trust score
6/10
MyAI rating
9.4
Runs
1
Freshness
Stale
Source-linked row with explicit verification status.
Tested on 2024-09-08; 718 days old.
Enterprise buying note
Retailer we'd check first
Amazon search plus reseller quotes
For datacenter and accelerator parts, Amazon is useful for spotting live listings, accessories, or used pulls, but serious procurement usually happens through integrators, brokers, or cloud partners.
Enterprise procurement, not retail
This silicon is typically acquired through an authorized OEM partner, system integrator, or hyperscaler reseller. For on-demand access, compare hourly rates at RunPod, Vast.ai, Lambda Labs, or your existing cloud provider before committing to capital expenditure.
This is one of the highest-scoring parts in our database, so it is worth tracking when inventory lands near fair-market pricing.
Affiliate note: we do not have a device-level ASIN yet, so this opens tagged Amazon search results for the exact product name.