stabilityai/stablelm-base-alpha-3b

`stabilityai/stablelm-base-alpha-3b` is a pre-trained language model available on the Hugging Face Hub. It's specifically designed for the `text-generation` task in the `transformers` library. If you…

azureFoundry/HuggingFace/stabilityai-stablelm-base-alpha-3b@6

`stabilityai/stablelm-base-alpha-3b` is a pre-trained language model available on the Hugging Face Hub. It's specifically designed for the `text-generation` task in the `transformers` library. If you want to learn more about the model's architecture, hyperparameters, limitations, and biases, you can find this information on the model's dedicated [Model Card on the Hugging Face Hub](https://huggingface.co/stabilityai/stablelm-base-alpha-3b).

Here's an example API request payload that you can use to obtain predictions from the model: ``` { "inputs": "My name is Julien and I like to" } ```

Specifications & capabilities

Input
text
Output
text
Published model ID
stablelm-base-alpha-3b

Evaluations

41–60 / 4,157

LLM-Perf

1,539,861.742

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
1
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
cfcf9fd1b37eccecc2626456b9bb28cf9c54400d1f3a6c9f030ac4d9eef5ff6c
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,539,861.742

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
1
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
394983500a45d1f3535bad3e900a81d0afac444735f8d44be3d638621119d920
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,527,304.003

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemv
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
0504a80f2abb8bc3117c94df6dc2a7dff8167c3992927cdcd1e89cde87b4bb92
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,527,304.003

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemv
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
561ce2c6d744f1f8c784b2e0d9284f519f5685e1d40f94cb615e2191d8ae631b
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,497,553.032

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
2a01c467e4f322acfd5131141d5de28f7ebd6a6f364e3c06377872ebe78764c2
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,497,553.032

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
e2060736c337adaf2d80e4e6c8a25daf255ca9c6309701b8f696928a32eaceff
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,494,570.495

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
74f5d0fc13ddbf39b9323c0212eb283179f8d7ade64cce6ba20dd1bf4192fe99
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,494,570.495

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
bb955b5e85260e229bb65aa08a14ca7c0375d30dd38222e2cb9d35987144e785
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,467,608.93

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemv
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
d2a026426fbf51e4c51ca20edb49729cb9e63177ee560f26a7feed9d41e58054
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,467,608.93

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemv
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
ffe24a8f251d285399ab4fc9aadd7e555dafb6c8a33c438b0cb1bb6c85ff8517
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,438,050.565

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
2
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
c54fe123730a3615ff3bdd86836934afa2a583f3a1fc2a588b86aba846d0a61e
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,438,050.565

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
2
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
0177d3c8e1d2015d1d85562fa291bfa75627a0b954d1df80a7c77bcdac59a158
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,304,380.266

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
1
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
653502d8442cd19137229abdabc598757552881e5dc0cf36e271ca9c15409a6e
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,304,380.266

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
1
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
380363a5d3fd0ad78cec389234523c9754d413b14dc8b49c4733545f6063767b
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,303,052.877

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemv
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
75127be8adaa3c8aa05bbcaa19a414152e86be703d40255181da0961d7e49e1e
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,303,052.877

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemv
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
6614421cf81c481d3c46c87f79da9c8a96ec3935371a0f9136398f225f759584
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,204,905.213

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
9399b4e31cd05125053e5dfebecb97265a0d039250525f0eb268726cc6a5706e
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,204,905.213

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
09dfdb66215e784e381da86c17a5e6a9aa7b76c0f51217258ee27221b84a2df9
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,200,118.375

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
a789d7bbafe7f0a38ade97478756dd3fe1bdbca0298c4a0cfc7589426338e083
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

LLM-Perf

1,200,118.375

source id
llmPerf
leaderboard name
decode efficiency value
dimension
runtime
metric
decode efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
gpt_neox
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.1
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
f258f23b293cf48099236ca904a49cb83bfa1774d4655b4582b132301b338dc4
complete_configuration_location
source record config and original CSV artifact
source model id
stabilityai/stablelm-base-alpha-3b
Original result ↗

Provider specifications & access

1–2 / 2

stabilityai/stablelm-base-alpha-3b ↗

source model id
HuggingFace/stabilityai-stablelm-base-alpha-3b@5
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
cc-by-sa-4.0
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    stabilityai/stablelm-base-alpha-3b ↗

    source model id
    HuggingFace/stabilityai-stablelm-base-alpha-3b@6
    hosting
    foundry-managed-compute
    platforms
    • Microsoft Foundry
    license
    cc-by-sa-4.0
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities

      Original sources

      1–8 / 8

      Hugging Face · Model repository ↗

      Microsoft Foundry · Model page ↗

      Microsoft Foundry · Model page ↗

      Microsoft Foundry · Model documentation ↗

      Hugging Face · Catalog ↗

      Hugging Face · Catalog ↗

      Hugging Face · Catalog ↗

      Microsoft Foundry · Catalog ↗

      Source links

      1–4 / 4

      Explore more models

      Models

      Loading
      EVIDENCE / COMPARISON

      Benchmarks

      Explore every benchmark

      Loading

      These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

      Install TensorTrace