LLM-Perf
Indexed results from LLM-Perf. Each ranking retains its own metric and evaluation conditions.
prefill energy cpu
llmPerf · LLM-Perf · prefill energy cpu · kilowattHours · Lower is better
decode memory max_process_vram
llmPerf · LLM-Perf · decode memory max_process_vram · megabytes · Lower is better
load latency p90
llmPerf · LLM-Perf · load latency p90 · seconds · Lower is better
- Qwen/Qwen1.5-1.8Bqwen7.042
- Qwen/Qwen1.5-0.5Bqwen7.095
- Qwen/Qwen1.5-7Bqwen7.145
- Qwen/Qwen1.5-4Bqwen7.216
prefill memory max_global_vram
llmPerf · LLM-Perf · prefill memory max_global_vram · megabytes · Lower is better
load memory max_reserved
llmPerf · LLM-Perf · load memory max_reserved · megabytes · Lower is better
decode memory max_allocated
llmPerf · LLM-Perf · decode memory max_allocated · megabytes · Lower is better
- EleutherAI/gpt-j-6beleutherai6,910.273
decode energy gpu
llmPerf · LLM-Perf · decode energy gpu · kilowattHours · Lower is better
load latency mean
llmPerf · LLM-Perf · load latency mean · seconds · Lower is better
per_token throughput value
llmPerf · LLM-Perf · per_token throughput value · tokensPerSecond · Higher is better
decode efficiency value
llmPerf · LLM-Perf · decode efficiency value · tokensPerKilowattHour · Higher is better
- distilbert/distilgpt2distilbert5,341,765.023
- openai-community/gpt2openai-community3,247,722.966
load memory max_ram
llmPerf · LLM-Perf · load memory max_ram · megabytes · Lower is better
- EleutherAI/gpt-j-6beleutherai3,812.012
decode energy ram
llmPerf · LLM-Perf · decode energy ram · kilowattHours · Lower is better
Models
LoadingBenchmarks
Explore every benchmark
LoadingThese charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.