VRAM
48 GB
TDP
350 W
MSRP
$7.8k
Perf/W
0.50 img/min/W
Cost/1K tok
$0.47/M
Tested
2024-06-11
NVIDIA L40S 48GB has a source-attributed result of 22.0 tok/s on Llama 3 70B Q4 (batch 1, 4096-token context, Q4_K_M; runtime not documented). This is a reference report, not an independently verified lab result. A 70B Q4_K_M artifact needs roughly 40GB for weights alone; smaller memory configurations require explicit offload or model splitting and do not establish full-GPU residency.
NVIDIA L40S 48GB — Ada Lovelace datacenter GPU with 48GB GDDR6 at 864 GB/s, 350W TDP. Passive cooling (requires server airflow). Positioned between consumer cards and H100 for inference-heavy workloads.
48GB can hold many 70B Q4 artifacts, with context limited by the remaining memory. 32B FP16 weights need about 64GB, so they do not fit fully. Passive cooling requires a compatible server with specified airflow.
NVIDIA L40S 48GB with 48GB VRAM at 350W TDP, scored across 9 workloads with 10 benchmark records.
Reference workload
SDXL image gen
18 img/min
Quantization
FP16
· batch 1
| Workload | Score | Quant | Context | σ | Status | Tested |
|---|---|---|---|---|---|---|
| Llama 3 8B FP16 llm | 175.0tok/s | FP16 | 4K | , | Curated Aggregate | 2024-06-11 |
| Llama 3 70B Q4 llm | 22.0tok/s | Q4_K_M | 4K | , | Curated Aggregate | 2024-08-07 |
| SDXL image gen image | 18.0img/min | FP16 | , | , | Curated Aggregate | 2026-04-29 |
| Embedding throughput embedding | 7400.0emb/s | FP16 | 1K | , | Curated Aggregate | 2024-08-26 |
| Mistral 7B llm | 142.0tok/s | Q4_K_M | 4K | , | Curated Aggregate | 2024-06-18 |
| Gemma 2 9B llm | 118.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2024-08-23 |
| Qwen 2.5 14B llm | 92.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2024-11-15 |
| DeepSeek-R1 7B llm | 138.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2025-02-20 |
| Asure 12B llm | 110.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2025-05-01 |
There is insufficient comparable evidence to score NVIDIA L40S 48GB. A score needs results for multiple devices with matching workload, runtime version, quantization, context and batch size.
Planning estimates for 48GB: weights plus at least 2GB or 10% overhead. Actual KV cache depends on model, context and cache format; confirm the artifact before buying.
Curated from public sources (llama.cpp logs / vendor specs / vLLM community)
View sourceHow we benchmark →Trust score
6/10
MyAI rating
Not scored
Runs
Not documented
Freshness
Aging
Source-linked row with explicit verification status.
Record date: 2026-04-29; 133 days old.
Enterprise buying note
Retailer we'd check first
Amazon search plus reseller quotes
For datacenter and accelerator parts, Amazon is useful for spotting live listings, accessories, or used pulls, but serious procurement usually happens through integrators, brokers, or cloud partners.
Enterprise procurement, not retail
This silicon is typically acquired through an authorized OEM partner, system integrator, or hyperscaler reseller. For on-demand access, compare hourly rates at RunPod, Vast.ai, Lambda Labs, or your existing cloud provider before committing to capital expenditure.
There is insufficient comparable evidence for a purchase recommendation. Check model fit, software support and a current seller quote.
Affiliate note: we do not have a device-level ASIN yet, so this opens tagged Amazon search results for the exact product name.