ASICGroq

Groq LPU (per chip)

Curated Aggregate·2025-01-057 workloads · 10 records
MyAI RatingNot scoredInsufficient comparison evidence

VRAM

0.23 GB

TDP

215 W

MSRP

$20k

Perf/W

3.49 tok/s/W

Cost/1K tok

$0.28/M

Tested

2025-01-05

Quick answer

How many tokens per second does Groq LPU (per chip) produce on Llama 3 70B Q4?

Groq LPU (per chip) has a source-attributed result of 285.0 tok/s on Llama 3 70B Q4 (batch 1, 8192-token context, FP16; runtime not documented). This is a reference report, not an independently verified lab result. A 70B Q4_K_M artifact needs roughly 40GB for weights alone; smaller memory configurations require explicit offload or model splitting and do not establish full-GPU residency.

Source: MyAIHardware benchmark database (bench-groq-l3-70b-q4)As of 2024-08-25

Verdict

Groq LPU (per chip) with 0.23GB VRAM at 215W TDP, scored across 7 workloads with 10 benchmark records.

Reference workload

Mistral 7B

1280 tok/s

Quantization

FP16

8K context · batch 1

LLM Inference Performance

0500100015002000Llama 3 8BFP16Llama 370B Q4Mistral 7BQwen 2.514BDeepSeek-R17BGemma 29B

Benchmarks (7 workloads)

WorkloadScoreQuantContextσStatusTested
Llama 3 8B FP16

llm

1850.0tok/sFP168K, Curated Aggregate2026-04-20
Llama 3 70B Q4

llm

285.0tok/sFP168K, Curated Aggregate2024-08-25
Mistral 7B

llm

1280.0tok/sFP168K, Curated Aggregate2026-05-04
Qwen 2.5 14B

llm

480.0tok/sFP168K, Curated Aggregate2024-08-22
DeepSeek-R1 7B

llm

620.0tok/sFP168K, Curated Aggregate2025-02-04
Gemma 2 9B

llm

580.0tok/sFP88K, Curated Aggregate2024-09-20
Asure 12B

llm

520.0tok/sFP88K, Curated Aggregate2025-05-01

MyAI Score: not scored

There is insufficient comparable evidence to score Groq LPU (per chip). A score needs results for multiple devices with matching workload, runtime version, quantization, context and batch size.

Workload Fit

Planning estimates for 0.23GB: weights plus at least 2GB or 10% overhead. Actual KV cache depends on model, context and cache format; confirm the artifact before buying.

Q4
Q8
FP16
7-8B
Won't fit
Won't fit
Won't fit
13-14B
Won't fit
Won't fit
Won't fit
32B
Won't fit
Won't fit
Won't fit
70B
Won't fit
Won't fit
Won't fit
Reported batch-one examples
Mistral 7B1280 tok/s
Llama 3 8B FP161850 tok/s
Asure 12B520 tok/s

Source

Groq vendor numbers.

View sourceHow we benchmark →

Public Trust Layer

Trust score

6/10

MyAI rating

Not scored

Runs

Not documented

Freshness

Aging

Source-linked row with explicit verification status.

Record date: 2026-05-04; 128 days old.

Open primary source

Enterprise buying note

Where to buy

Reseller compare

Retailer we'd check first

Amazon search plus reseller quotes

For datacenter and accelerator parts, Amazon is useful for spotting live listings, accessories, or used pulls, but serious procurement usually happens through integrators, brokers, or cloud partners.

Enterprise procurement, not retail

This silicon is typically acquired through an authorized OEM partner, system integrator, or hyperscaler reseller. For on-demand access, compare hourly rates at RunPod, Vast.ai, Lambda Labs, or your existing cloud provider before committing to capital expenditure.

  • +Use $20k only as a rough anchor. Enterprise street pricing moves with supply, warranty, and included accessories.
  • +Confirm cooling, power delivery, and return terms before you purchase. These parts often ship without consumer-friendly safeguards.
  • +If this is for production, compare against authorized reseller quotes before you commit.

There is insufficient comparable evidence for a purchase recommendation. Check model fit, software support and a current seller quote.

Affiliate note: we do not have a device-level ASIN yet, so this opens tagged Amazon search results for the exact product name.

Same Tier

Cerebras WSE-3 (CS-3)
44GB
AWS Trainium2
96GB
Google TPU v5p
95GB
Google TPU v5e
16GB