LLM-Perf

Indexed results from LLM-Perf. Each ranking retains its own metric and evaluation conditions.

load energy ram

llmPerf · LLM-Perf · load energy ram · kilowattHours · Lower is better

  1. Qwen/Qwen2-beta-14Bqwen0

prefill energy gpu

llmPerf · LLM-Perf · prefill energy gpu · kilowattHours · Lower is better

  1. google/recurrentgemma-2bgoogle0

prefill latency mean

llmPerf · LLM-Perf · prefill latency mean · seconds · Lower is better

  1. bigscience/bloomz-560mbigscience0.023

prefill latency p99

llmPerf · LLM-Perf · prefill latency p99 · seconds · Lower is better

  1. Qwen/Qwen1.5-1.8Bqwen0.098
  2. Qwen/Qwen1.5-0.5Bqwen0.098
  3. Qwen/Qwen1.5-7Bqwen0.131
  4. Qwen/Qwen1.5-4Bqwen0.161

per_token latency p99

llmPerf · LLM-Perf · per_token latency p99 · seconds · Lower is better

  1. tiiuae/falcon-rw-1btiiuae0.017

decode latency p90

llmPerf · LLM-Perf · decode latency p90 · seconds · Lower is better

  1. gemma-2bgoogle1.474
  2. gemma-7bgoogle2.287

decode latency p99

llmPerf · LLM-Perf · decode latency p99 · seconds · Lower is better

  1. meta-llama/Meta-Llama-3-8B-Instructmeta-llama2.169
  2. LLaMA3_8Bmeta-llama2.198

decode memory max_global_vram

llmPerf · LLM-Perf · decode memory max_global_vram · megabytes · Lower is better

  1. LLaMA3_8Bmeta-llama6,546.784
  2. meta-llama/Meta-Llama-3-8B-Instructmeta-llama6,546.784

load energy cpu

llmPerf · LLM-Perf · load energy cpu · kilowattHours · Lower is better

  1. gemma-2bgoogle0
  2. gemma-7bgoogle0

decode efficiency value

llmPerf · LLM-Perf · decode efficiency value · tokensPerKilowattHour · Higher is better

  1. EleutherAI/pythia-70meleutherai14,962,849.687
  2. EleutherAI/pythia-160meleutherai7,176,966.332
  3. EleutherAI/pythia-410meleutherai3,124,342.246
  4. EleutherAI/pythia-1.4beleutherai1,243,893.526
  5. EleutherAI/pythia-1.4beleutherai1,233,153.993
  6. EleutherAI/pythia-1.3beleutherai1,228,346.842
  7. EleutherAI/pythia-1.3beleutherai1,228,285.82
  8. EleutherAI/pythia-2.7beleutherai692,415.471
  9. EleutherAI/pythia-6.7beleutherai300,407.429
  10. EleutherAI/pythia-12beleutherai184,132.587
  11. EleutherAI/polyglot-ko-12.8beleutherai166,750.257
  12. EleutherAI/polyglot-ko-12.8beleutherai165,216.415

decode latency p50

llmPerf · LLM-Perf · decode latency p50 · seconds · Lower is better

  1. gemma-2bgoogle2.785

load latency p90

llmPerf · LLM-Perf · load latency p90 · seconds · Lower is better

  1. Deci/DeciLM-7Bdeci10.264

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace