VRAM
141 GB
TDP
700 W
MSRP
$30k
Perf/W
0.12 tok/s/W
Cost/1K tok
$0.0038/k
Tested
2024-04-06
NVIDIA H200 141GB has a source-attributed result of 84.0 tok/s on Llama 3 70B Q4 (batch 1, 8192-token context, Q4_K_M; runtime not documented). This is a reference report, not an independently verified lab result. A 70B Q4_K_M artifact needs roughly 40GB for weights alone; smaller memory configurations require explicit offload or model splitting and do not establish full-GPU residency.
NVIDIA H200 141GB — Hopper-generation capacity-upgrade GPU. 141GB HBM3e at 4.8 TB/s, 700W TDP. Same compute dies as H100 with dramatically more and faster memory. Purpose-built for large-model inference.
141GB supports large quantized models, but 405B FP8 weights require about 405GB before overhead. Even 70B FP16 leaves little space in 141GB for cache and runtime buffers. Check the actual artifact and parallel configuration.
The capacity king. Right answer when 80GB isn't enough and you need 141GB. For everything else, H100 or L40S are more economical.
Large quantized models that fit after runtime and context allocations; 405B FP8 does not fit a single 141GB device.
HBM3e thermal management at sustained 4.8 TB/s bandwidth — requires proper datacenter airflow. Memory errors under thermal stress more common than H100 due to higher density.
Ubuntu 22.04/24.04 LTS (reference), RHEL 9
700W SXM. Datacenter infrastructure required. The 141GB HBM3e draws more board power than H100 under sustained memory load.
NVIDIA H200 141GB with 141GB VRAM at 700W TDP, scored across 11 workloads with 14 benchmark records.
Reference workload
DeepSeek-R1 7B
305 tok/s
Quantization
Q4_K_M
8K context · batch 1
| Workload | Score | Quant | Context | σ | Status | Tested |
|---|---|---|---|---|---|---|
| Llama 3 70B Q4 llm | 84.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2024-04-06 |
| Llama 3 70B Q8 llm | 58.0tok/s | Q8_0 | 8K | , | Curated Aggregate | 2024-04-10 |
| Llama 3 8B FP16 llm | 340.0tok/s | FP16 | 4K | , | Curated Aggregate | 2025-11-18 |
| Mistral 7B llm | 305.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2026-01-28 |
| SDXL image gen image | 36.0img/min | FP16 | , | , | Curated Aggregate | 2026-02-15 |
| DeepSeek-R1 7B llm | 305.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2026-04-20 |
| Gemma 2 9B llm | 220.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2024-07-30 |
| Qwen 2.5 14B llm | 178.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2024-12-15 |
| Llama 3 8B Q4 llm | 215.0tok/s | Q4_K_M | 128K | , | Curated Aggregate | 2025-02-04 |
| Embedding throughput embedding | 15200.0emb/s | FP16 | 1K | , | Curated Aggregate | 2024-09-08 |
| Asure 12B llm | 230.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2025-05-01 |
There is insufficient comparable evidence to score NVIDIA H200 141GB. A score needs results for multiple devices with matching workload, runtime version, quantization, context and batch size.
Planning estimates for 141GB: weights plus at least 2GB or 10% overhead. Actual KV cache depends on model, context and cache format; confirm the artifact before buying.
Trust score
6/10
MyAI rating
Not scored
Runs
Not documented
Freshness
Aging
Source-linked row with explicit verification status.
Record date: 2026-04-20; 142 days old.
Enterprise buying note
Retailer we'd check first
Amazon search plus reseller quotes
For datacenter and accelerator parts, Amazon is useful for spotting live listings, accessories, or used pulls, but serious procurement usually happens through integrators, brokers, or cloud partners.
Enterprise procurement, not retail
This silicon is typically acquired through an authorized OEM partner, system integrator, or hyperscaler reseller. For on-demand access, compare hourly rates at RunPod, Vast.ai, Lambda Labs, or your existing cloud provider before committing to capital expenditure.
There is insufficient comparable evidence for a purchase recommendation. Check model fit, software support and a current seller quote.
Affiliate note: we do not have a device-level ASIN yet, so this opens tagged Amazon search results for the exact product name.