VRAM
48 GB
TDP
350 W
MSRP
$7.8k
Perf/W
0.50 emb/s/W
Cost/1K tok
$0.47/M
Tested
2024-06-11
NVIDIA L40S 48GB produces approximately 22.0 tok/s on Llama 3 70B at Q4_K_M quantization (batch 1, 4096-token context, 48GB VRAM, 350W TDP). That figure comes from 1 measured run on llama.cpp. The 70B model needs roughly 40GB of VRAM at Q4, so headroom and KV cache budget matter as much as raw throughput.
NVIDIA L40S 48GB — Ada Lovelace datacenter GPU with 48GB GDDR6 at 864 GB/s, 350W TDP. Passive cooling (requires server airflow). Positioned between consumer cards and H100 for inference-heavy workloads.
48GB VRAM comfortably hosts 70B Q4 with large context or 32B FP16 full-GPU. The sweet spot for single-GPU enterprise inference where H100 is overkill. ~110 tok/s on 12B Q4. Passive cooling means datacenter deployment only — not a desktop card. Best for: enterprise inference serving, RAG pipelines, and multi-tenant LLM hosting where 48GB is the right size.
NVIDIA L40S 48GB with 48GB VRAM at 350W TDP, scored across 9 workloads with 10 benchmark records.
Best workload
Embedding throughput
10500 emb/s
Quantization
FP16
1K context · batch 1
| Workload | Score | Quant | Context | σ | Status | Tested |
|---|---|---|---|---|---|---|
| Llama 3 8B FP16 llm | 175.0tok/s | FP16 | 4K | , | Curated Aggregate | 2024-06-11 |
| Llama 3 70B Q4 llm | 22.0tok/s | Q4_K_M | 4K | , | Curated Aggregate | 2024-08-07 |
| SDXL image gen image | 18.0img/min | FP16 | , | , | Curated Aggregate | 2026-04-29 |
| Embedding throughput embedding | 10500.0emb/s | FP16 | 1K | , | Curated Aggregate | 2024-08-04 |
| Mistral 7B llm | 142.0tok/s | Q4_K_M | 4K | , | Curated Aggregate | 2024-06-18 |
| Gemma 2 9B llm | 118.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2024-08-23 |
| Qwen 2.5 14B llm | 92.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2024-11-15 |
| DeepSeek-R1 7B llm | 138.0tok/s | Q4_K_M | 8K | , | Curated Aggregate | 2025-02-20 |
| Asure 12B llm | 110.0tok/s | Q4_K_M | 8K | ±5.5 | Curated Aggregate | 2025-05-01 |
NVIDIA L40S 48GB clears a 9.0/10 based on workload-normalized throughput, memory headroom, efficiency, value, trust, and coverage.
What models fit this 48GB card at different quantization levels.
Trust score
6/10
MyAI rating
9.0
Runs
1
Freshness
Stale
Source-linked row with explicit verification status.
Tested on 2024-08-04; 753 days old.
Enterprise buying note
Retailer we'd check first
Amazon search plus reseller quotes
For datacenter and accelerator parts, Amazon is useful for spotting live listings, accessories, or used pulls, but serious procurement usually happens through integrators, brokers, or cloud partners.
Enterprise procurement, not retail
This silicon is typically acquired through an authorized OEM partner, system integrator, or hyperscaler reseller. For on-demand access, compare hourly rates at RunPod, Vast.ai, Lambda Labs, or your existing cloud provider before committing to capital expenditure.
This is one of the highest-scoring parts in our database, so it is worth tracking when inventory lands near fair-market pricing.
Affiliate note: we do not have a device-level ASIN yet, so this opens tagged Amazon search results for the exact product name.