CPU-onlyAMD

AMD Threadripper PRO 7995WX (96-core)

Curated Aggregate·2024-09-224 workloads · 6 records
MyAI RatingNot scoredInsufficient comparison evidence

VRAM

0 GB

TDP

350 W

MSRP

$10.0k

Perf/W

0.03 tok/s/W

Cost/1K tok

$0.0088/k

Tested

2024-09-22

Quick answer

How many tokens per second does AMD Threadripper PRO 7995WX (96-core) produce on Llama 3 70B Q4?

AMD Threadripper PRO 7995WX (96-core) has a source-attributed result of 2.4 tok/s on Llama 3 70B Q4 (batch 1, 4096-token context, Q4_K_M; runtime not documented). This is a reference report, not an independently verified lab result. A 70B Q4_K_M artifact needs roughly 40GB for weights alone; smaller memory configurations require explicit offload or model splitting and do not establish full-GPU residency.

Source: MyAIHardware benchmark database (bench-tr-7995wx-l3-70b-q4)As of 2024-10-04

Verdict

AMD Threadripper PRO 7995WX (96-core) with 0GB VRAM at 350W TDP, scored across 4 workloads with 6 benchmark records.

Reference workload

Mistral 7B

14 tok/s

Quantization

Q4_K_M

4K context · batch 1

LLM Inference Performance

07142128Llama 3 8BQ4Llama 370B Q4Phi-3 MiniMistral 7B

Benchmarks (4 workloads)

WorkloadScoreQuantContextσStatusTested
Llama 3 8B Q4

llm

12.0tok/sQ4_K_M4K, Curated Aggregate2024-09-22
Llama 3 70B Q4

llm

2.4tok/sQ4_K_M4K, Curated Aggregate2024-10-04
Phi-3 Mini

llm

28.0tok/sQ4_K_M4K, Curated Aggregate2024-10-18
Mistral 7B

llm

14.0tok/sQ4_K_M4K, Curated Aggregate2024-11-05

MyAI Score: not scored

There is insufficient comparable evidence to score AMD Threadripper PRO 7995WX (96-core). A score needs results for multiple devices with matching workload, runtime version, quantization, context and batch size.

Workload Fit

Planning estimates for 0GB: weights plus at least 2GB or 10% overhead. Actual KV cache depends on model, context and cache format; confirm the artifact before buying.

Q4
Q8
FP16
7-8B
Won't fit
Won't fit
Won't fit
13-14B
Won't fit
Won't fit
Won't fit
32B
Won't fit
Won't fit
Won't fit
70B
Won't fit
Won't fit
Won't fit
Reported batch-one examples
Mistral 7B14 tok/s
Phi-3 Mini28 tok/s
Llama 3 70B Q42 tok/s

Source

Pure CPU, no GPU offload.

View sourceHow we benchmark →

Public Trust Layer

Trust score

6/10

MyAI rating

Not scored

Runs

Not documented

Freshness

Stale

Source-linked row with explicit verification status.

Record date: 2024-11-05; 673 days old.

Open primary source

Best place to start

Where to buy

Search first

Retailer we'd check first

Amazon search

For mainstream AI hardware, Amazon usually updates street pricing, seller availability, and shipping speed faster than most comparison sites.

Search Amazon listings
  • +Use $10.0k as your price anchor unless the part is clearly supply-constrained or newly launched.
  • +Check the exact cooler, board partner, or memory configuration before buying. The silicon may match, but noise and thermals do not.
  • +If the Amazon price looks inflated, wait or compare against recent street pricing rather than paying a panic premium.

There is insufficient comparable evidence for a purchase recommendation. Check model fit, software support and a current seller quote.

Affiliate note: we do not have a device-level ASIN yet, so this opens tagged Amazon search results for the exact product name.

Same Tier

Intel Xeon 6980P (128c Granite Rapids)
0GB
Intel Core Ultra 9 285K
0GB