LLM-Perf

Indexed results from LLM-Perf. Each ranking retains its own metric and evaluation conditions.

prefill energy cpu

llmPerf · LLM-Perf · prefill energy cpu · kilowattHours · Lower is better

  1. google/recurrentgemma-9bgoogle0

decode memory max_process_vram

llmPerf · LLM-Perf · decode memory max_process_vram · megabytes · Lower is better

  1. EleutherAI/gpt-j-6beleutherai0

load latency p90

llmPerf · LLM-Perf · load latency p90 · seconds · Lower is better

  1. Qwen/Qwen1.5-1.8Bqwen7.042
  2. Qwen/Qwen1.5-0.5Bqwen7.095
  3. Qwen/Qwen1.5-7Bqwen7.145
  4. Qwen/Qwen1.5-4Bqwen7.216

prefill memory max_global_vram

llmPerf · LLM-Perf · prefill memory max_global_vram · megabytes · Lower is better

  1. gemma-2bgoogle6,804.734
  2. gemma-7bgoogle19,307.954

load memory max_reserved

llmPerf · LLM-Perf · load memory max_reserved · megabytes · Lower is better

  1. distilbert/distilgpt2distilbert113.246
  2. openai-community/gpt2openai-community148.898

decode memory max_allocated

llmPerf · LLM-Perf · decode memory max_allocated · megabytes · Lower is better

  1. EleutherAI/gpt-j-6beleutherai6,910.273

decode energy gpu

llmPerf · LLM-Perf · decode energy gpu · kilowattHours · Lower is better

  1. Qwen/Qwen1.5-0.5Bqwen0
  2. Qwen/Qwen1.5-1.8Bqwen0
  3. Qwen/Qwen1.5-4Bqwen0

load latency mean

llmPerf · LLM-Perf · load latency mean · seconds · Lower is better

  1. EleutherAI/pythia-1.4beleutherai40.038

per_token throughput value

llmPerf · LLM-Perf · per_token throughput value · tokensPerSecond · Higher is better

  1. EleutherAI/gpt-neo-125meleutherai74.757
  2. EleutherAI/gpt-neo-1.3Beleutherai38.488
  3. EleutherAI/gpt-neo-2.7Beleutherai28.952

decode efficiency value

llmPerf · LLM-Perf · decode efficiency value · tokensPerKilowattHour · Higher is better

  1. distilbert/distilgpt2distilbert5,341,765.023
  2. openai-community/gpt2openai-community3,247,722.966

load memory max_ram

llmPerf · LLM-Perf · load memory max_ram · megabytes · Lower is better

  1. EleutherAI/gpt-j-6beleutherai3,812.012

decode energy ram

llmPerf · LLM-Perf · decode energy ram · kilowattHours · Lower is better

  1. bigscience/bloomz-560mbigscience0

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace