EleutherAI/pythia-2.7b

Explore EleutherAI/pythia-2.7b: model specifications, capabilities, benchmark results, platform pricing and original sources.

huggingface/EleutherAI/pythia-2.7b

Specifications & capabilities

Input
text
Output
text
Published model ID
pythia-2.7b

Evaluations

1–20 / 11,771

LLM-Perf

1,261.371

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float32
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
6
benchmark_evaluation
true
comparison_configuration_sha256
a34e18f8fc27f763bf65a5538d6c7e3ed59b3cb39685c56c800785e1200cff82
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

1,261.371

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float32
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
6
benchmark_evaluation
true
comparison_configuration_sha256
cb9bc7ac3ed27754de6ca9b0e280506cab43632ceecb9baba899580fdfb7bc2a
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

1,215.173

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
53b0dccd791248f5fd46f91b126deae98f1c33f1ef17ea4c3cc95b873e5bd35b
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

1,215.173

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-unquantized-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-unquantized-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
6522994bdc70e80ee190f9011ed4b1977eb99edd862c0a6a0914a4e4be778aac
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

1,097.921

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.1+cu121
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float32
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.75552
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.21.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.33.0
config.environment.transformers_version
4.43.4
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
6
benchmark_evaluation
true
comparison_configuration_sha256
5cb962f1221aadb42e2b515085a17c854328b345e290a3baf79c7b4e4c669c36
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

1,097.921

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-unquantized-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.1+cu121
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float32
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.75552
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.21.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.33.0
config.environment.transformers_version
4.43.4
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-unquantized-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
6
benchmark_evaluation
true
comparison_configuration_sha256
b9b625f72199e891e9308e5595807bfb5541013ecc389a39306705bc486981a5
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
load memory max_process_vram
dimension
runtime
metric
load memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
2
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
load memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
697970535bce6393c73ce6a60e35767ad87dcad65283596f6a246b568dad9403
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
load memory max_process_vram
dimension
runtime
metric
load memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
load memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
7
benchmark_evaluation
true
comparison_configuration_sha256
883c7b5d105031772728335dabf38fed2012aa8237a117f1759f624e38cf83f0
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.0
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
2
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
da7c4518f1344c885448e3ac0b176f1ef20be7b3031092be053e24870b1e1764
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.0+cu121
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.29792
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.10.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.19.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.30.1
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.40.2
config.scenario.input_shapes.batch_size
1
config.backend.hub_kwargs.force_download
False
config.scenario.input_shapes.num_choices
2
config.backend.hub_kwargs.local_files_only
False
config.backend.quantization_config.version
2
config.backend.hub_kwargs.trust_remote_code
True
config.environment.optimum_benchmark_version
0.2.0
config.scenario.input_shapes.sequence_length
256
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
model_revision
main
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
14
benchmark_evaluation
true
comparison_configuration_sha256
f508b21253c978c0be8c991a2a6d151044e1aaee783cca49fe4d1b8a5b6819bc
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
3aac1f752bc89716192dee9cd163d2c62224418efe4c7f73bda94291a173a814
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
load memory max_process_vram
dimension
runtime
metric
load memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.1+cu121
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.75552
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.21.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.33.0
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.43.4
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
2
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
load memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
7132c0fd9178855d6a7bcd414a6e521ad7d46cd7169ef5d2de4c7b51944d9f13
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
decode memory max_process_vram
dimension
runtime
metric
decode memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
decode memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
cf3026403b2f98b5bee56a9cd7a8f4bdc39f6f33db53e911325d3f39063f90f3
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
load memory max_process_vram
dimension
runtime
metric
load memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.1+cu121
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.75552
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.33.0
config.environment.transformers_version
4.43.4
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
load memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
907102ef3432c9a9f668c1d0a080fa12509cf05d07936e307b28c4f3f14413f6
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemv
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
0c1ab59185dd8dd27a12d23574fb49975dd7dfadf77313866fc89a114a00e9cc
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
056162b8956c649ecaee6caa3d78cea630c9197cc7fc543de5c6d1db2ab40708
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
load memory max_process_vram
dimension
runtime
metric
load memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemm
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
load memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
8c6a893bdb5b171521bdac0dd057366803ebe1877015614d7e099966e0ab1b56
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
decode energy gpu
dimension
runtime
metric
decode energy gpu
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-unquantized-c7i.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cpu
config.backend.library
transformers
config.backend.version
2.4.0+cpu
config.environment.cpu
Intel(R) Xeon(R) Platinum 8488C
config.backend.eval_mode
True
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
96
config.environment.processor
x86_64
config.environment.cpu_ram_mb
199386.689536
config.backend.autocast_enabled
False
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.0
config.environment.transformers_version
4.43.4
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode energy gpu
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-unquantized-c7i.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
6
benchmark_evaluation
true
comparison_configuration_sha256
bd57338c1009a776f28be4dfb62df9095f71cdca734296b1b9fc3c4d4e148dbf
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
2
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
8caa7caac859730da4e4791e5dbdafe239f600ecb4e03c2be2036bae4233e04c
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.1+cu121
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.75552
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.33.0
config.environment.transformers_version
4.43.4
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
10
benchmark_evaluation
true
comparison_configuration_sha256
cac8a5be5bf6d8197c479032c45b5c1a81f698151648000f9dd79a1c3e481cdc
complete_configuration_location
source record config and original CSV artifact
source model id
EleutherAI/pythia-2.7b
Original result ↗

Original sources

1–3 / 3

Hugging Face · EleutherAI/pythia-2.7b ↗

Hugging Face · Catalog ↗

Hugging Face · Catalog ↗

Source links

1–2 / 2

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace