llm workload · 22 runs on record

Llama 3 8B (FP16)

Meta Llama 3 8B Instruct, full FP16 weights, batch 1, 2K context.

Primary metric: Tokens / sec (tok/s)

Reference prompts

Representative prompts for this workload. Exact prompts and harness settings still depend on the cited source for each record.

  • Prompt 1

    Explain the relationship between attention heads and KV cache memory usage for a 7B-parameter transformer at 4K context.

  • Prompt 2

    Write a 60-word product description for a mid-range AI workstation: 1× RTX 4090, 64GB DDR5, 2TB NVMe. Highlight one trade-off.

  • Prompt 3

    List five concrete differences between INT4 weight-only quantization (Q4_K_M) and INT8 quantization (Q8_0) for inference.

  • Prompt 4

    I have 12GB of VRAM and want to run a coding assistant locally. What model size and quantization fit, with what context length?

  • Prompt 5

    Summarize the difference between greedy decoding and nucleus sampling in 3 short bullet points.

Reference runtime command

A representative invocation for reproducing this workload class. Source-specific runs may use adjacent runtimes unless the record says otherwise.

shell
llama-server -m llama-3-8b-instruct.fp16.gguf -c 4096 -ngl 999 --seed 42 --prompt-file prompts.txt

Single batch. FP16 weights, ~16GB VRAM. Median of 5 runs, warm-up discarded.

Full leaderboard

Every record for Llama 3 8B FP16.

No comparable multi-device results

The current selection lacks two devices with matching workload, quantization, context, batch and documented runtime. No speed winner is assigned. Inspect the source-attributed records.

Cite this benchmark

Use this in your paper, blog post, or comparison table.

BibTeX
@misc{myaihardware_llama3-8b-fp16_2026,
  title  = {MyAI Bench: Llama 3 8B (FP16)},
  author = {{MyAIHardware Contributors}},
  year   = {2026},
  url    = {https://www.myaihardware.com/benchmarks/workload/llama3-8b-fp16},
  note   = {Version 1.3, accessed 2026-09-09}
}
APA
MyAIHardware Contributors. (2026). MyAI Bench: Llama 3 8B (FP16). MyAIHardware. Retrieved 2026-09-09, from https://www.myaihardware.com/benchmarks/workload/llama3-8b-fp16
MLA
MyAIHardware Contributors. "MyAI Bench: Llama 3 8B (FP16)." MyAIHardware, 2026, https://www.myaihardware.com/benchmarks/workload/llama3-8b-fp16. Accessed 2026-09-09.
Plain text
MyAI Bench, Llama 3 8B (FP16). MyAIHardware Contributors, 2026. Version 1.3. https://www.myaihardware.com/benchmarks/workload/llama3-8b-fp16 (accessed 2026-09-09).