LLM-Perf
Indexed results from LLM-Perf. Each ranking retains its own metric and evaluation conditions.
load energy ram
llmPerf · LLM-Perf · load energy ram · kilowattHours · Lower is better
prefill energy gpu
llmPerf · LLM-Perf · prefill energy gpu · kilowattHours · Lower is better
prefill latency mean
llmPerf · LLM-Perf · prefill latency mean · seconds · Lower is better
prefill latency p99
llmPerf · LLM-Perf · prefill latency p99 · seconds · Lower is better
- Qwen/Qwen1.5-1.8Bqwen0.098
- Qwen/Qwen1.5-0.5Bqwen0.098
- Qwen/Qwen1.5-7Bqwen0.131
- Qwen/Qwen1.5-4Bqwen0.161
per_token latency p99
llmPerf · LLM-Perf · per_token latency p99 · seconds · Lower is better
decode latency p90
llmPerf · LLM-Perf · decode latency p90 · seconds · Lower is better
decode latency p99
llmPerf · LLM-Perf · decode latency p99 · seconds · Lower is better
decode memory max_global_vram
llmPerf · LLM-Perf · decode memory max_global_vram · megabytes · Lower is better
- LLaMA3_8Bmeta-llama6,546.784
- meta-llama/Meta-Llama-3-8B-Instructmeta-llama6,546.784
load energy cpu
llmPerf · LLM-Perf · load energy cpu · kilowattHours · Lower is better
decode efficiency value
llmPerf · LLM-Perf · decode efficiency value · tokensPerKilowattHour · Higher is better
- EleutherAI/pythia-70meleutherai14,962,849.687
- EleutherAI/pythia-160meleutherai7,176,966.332
- EleutherAI/pythia-410meleutherai3,124,342.246
- EleutherAI/pythia-1.4beleutherai1,243,893.526
- EleutherAI/pythia-1.4beleutherai1,233,153.993
- EleutherAI/pythia-1.3beleutherai1,228,346.842
- EleutherAI/pythia-1.3beleutherai1,228,285.82
- EleutherAI/pythia-2.7beleutherai692,415.471
- EleutherAI/pythia-6.7beleutherai300,407.429
- EleutherAI/pythia-12beleutherai184,132.587
- EleutherAI/polyglot-ko-12.8beleutherai166,750.257
- EleutherAI/polyglot-ko-12.8beleutherai165,216.415
decode latency p50
llmPerf · LLM-Perf · decode latency p50 · seconds · Lower is better
load latency p90
llmPerf · LLM-Perf · load latency p90 · seconds · Lower is better
- Deci/DeciLM-7Bdeci10.264
Models
LoadingBenchmarks
Explore every benchmark
LoadingThese charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.