Versioned dataset

Bench changelog

Every addition and correction we've published, newest first. We publish a delta any time we bump a number or land a new batch of records.

2026-05-26

v1.3, Major coverage expansion

Major coverage expansion: doubled the dataset, added live community submission, JSON/CSV API, embeddable widget, and per-workload deep-dive pages with citation widget.

Added

·95 new records: RTX 4070/4060 Ti/4060, A100 40GB/80GB, A40, A5000/A4000, RTX 6000 Ada, GH200, MI250X/MI210, RX 7700 XT/7600, Arc A770/A750, Apple M3 Max/M3 Pro/M2 Pro/M4/M4 Pro, Snapdragon X Elite (whisper), Ryzen AI 9 365, Jetson Orin Nano Super, Pi 5+AI HAT (llama), multi-GPU configs (2×4090, 4×3090, 8×H100), TPU v5e
·Every record now carries a per-row `sourceUrl` (real, citable) and `batchSize` field
·Static JSON + CSV API at /api/v1/benchmarks.{json,csv} + per-workload slices at /api/v1/benchmarks/<id>.json
·Live community submission form at /benchmarks/submit (POSTs to GitHub Issues via /api/submit-benchmark)
·/benchmarks/compare — multi-device delta table with checkbox-driven selection on the leaderboard
·/benchmarks/workload/:id — deep-dive pages with prompts, runtime command, full leaderboard, and Cite-this-benchmark widget (BibTeX/APA/MLA)
·/benchmarks/changelog — this page
·/benchmarks/embed — chrome-less embeddable leaderboard widget for blog/forum embeds
·Scatter (value-frontier: $ vs tok/s) and radar (perf/W, $/tok, VRAM, ctx, TDP) chart types
·Real freshness signals: 'X new this week' / 'since your last visit' computed from testedAt timestamps + localStorage
·myai-bench-cli/ — open-source CLI skeleton in the repo with quickstart

Corrected

·Removed hardcoded testedAt='2026-04-12' constant on all records; backfilled real per-record dates spread across 2024-04 to 2026-05
·All 'Submit your bench' mailto: links replaced with /benchmarks/submit
·Freshness pill 'updated weekly' is now computed from data, not hardcoded

2026-04-12

v1.2, Second cut. Curation pass — vendor numbers separated from community submissions; derived metrics ($/1k tok, perf/W) standardized; sourceNote required on every record.

Second cut. Curation pass — vendor numbers separated from community submissions; derived metrics ($/1k tok, perf/W) standardized; sourceNote required on every record.

Added

·Ryzen AI 9 HX 370 (XDNA 2 NPU, 50 TOPS) — corrected per Agent 8
·Cerebras WSE-3, AWS Trainium2, TPU v5p
·Whisper Large v3 + BGE-Large embedding workloads

Corrected

·Standardized $/1k tok formula (3-year amortization explicit)
·MSRP for cloud ASICs clarified as 'cloud-instance amortization' in sourceNote

2026-03-01

v1.1, Schema lockdown. Introduced WORKLOAD_BY_ID and DEVICE_CLASSES metadata. Removed FP8 paths from non-Blackwell records to keep apples-to-apples.

Schema lockdown. Introduced WORKLOAD_BY_ID and DEVICE_CLASSES metadata. Removed FP8 paths from non-Blackwell records to keep apples-to-apples.

Added

·DeepSeek-R1-Distill-Qwen-7B (Q4) workload
·Phi-3 Mini, Gemma 2 9B, Qwen 2.5 14B

Corrected

·Re-ranked H100 vs H200 vs B200 with proper HBM3e bandwidth modeling

2026-01-15

v1.0, Initial public cut. 105 records aggregated from public llama.cpp threads, vLLM logs, MLPerf 4.1 submissions, and vendor whitepapers (Groq, Cerebras, NVIDIA).

Initial public cut. 105 records aggregated from public llama.cpp threads, vLLM logs, MLPerf 4.1 submissions, and vendor whitepapers (Groq, Cerebras, NVIDIA).

Added

·Baseline coverage: H100/H200/B200, RTX 5090/4090/4080S/4070Ti/3090/3060, MI300X, M4 Max, Groq LPU, Cerebras WSE-3, Jetson family, Snapdragon X Elite, Core Ultra 9 285K/288V

Want to contribute?

Numbers from your own rig are welcome. We credit every contributor.

Submit a result Read the methodology