Deci/DeciLM-7B

Explore Deci/DeciLM-7B: model specifications, capabilities, benchmark results, platform pricing and original sources.

huggingface/Deci/DeciLM-7B

Specifications & capabilities

Input
text
Output
text
Published model ID
DeciLM-7B

Evaluations

1–20 / 9,064

LLM-Perf

44,855,245.622

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
db0024a3f31cead2bb3bc53b4eef99b1118e40a404250e5877f67ec4eb9c0be9
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

44,855,245.622

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
bc9e89f051cbc789b8e2147f8208f305a073dc78bcbfa1c086007a1e5aa8ce91
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

44,815,676.625

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
ec90a2fb70cc9f44bf608d5103f782ce029301c2efbacc4f0618cd2f68552996
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

44,815,676.625

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
85bcbde03a904021352981edae6168b3f6cd00339d215e88faaa34b8a8853ec3
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

39,774,188.331

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
7828665038ed517a6857f9a786768f4a921ba63ce37eafebcd471e6147ca4d11
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

39,774,188.331

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
6ce6bd128a368df283c5411452134b4a2638fa7cc1c09b84c490054a5407af63
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

39,237,590.22

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.1+cu121
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.75552
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.33.0
config.environment.transformers_version
4.43.4
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
2f226598eb21324ffc0f612c1eed98663091531b37b2a0db8561fab1ec70313d
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

39,237,590.22

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.1+cu121
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.75552
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.33.0
config.environment.transformers_version
4.43.4
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
707f3fcca1ceb3e8f9e9ec0f9cb50c5df3bbede2935b202ee0e7ed0b2d3393cd
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

39,219,476.916

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.1+cu121
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.75552
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.33.0
config.environment.transformers_version
4.43.4
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
649930eed47aebf85c12c28d27538800c625d172652c4d2db2371b4d015b5369
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

39,219,476.916

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.1+cu121
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.75552
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.33.0
config.environment.transformers_version
4.43.4
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
da90c132d9c6e74e60a6334cafab551f88cd4dd3cf8cac884ca7ce737a1c8954
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

38,726,247.073

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
2f3364363280b64ecb9bec2c9fd0c22f4c402cc5a6223d82aa5d8f0cc25b34ce
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

38,726,247.073

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
b5f72c5b5e172e483809467bc9c33357efa4dda7ebdaae8e24ffc5af8bd5dc5e
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

34,815,703.196

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
f8835caf67321fa104ee58c2d5154147c37130a1cf0bc9d4b9156e9e16a1ce28
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

34,815,703.196

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
003f6689d7ffb83a996cb312498f69d1d12170e5381967765eee5d06b79b6c88
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

34,379,500.033

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
07f9ca964e4a295d312c4ac3c1f43dcb5ab3f24c69b91af9531ed348efdfc19a
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

34,379,500.033

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
d77bf890f7248b52940f9061b9f94a03203c7eb218196f14b4c20eaa1f7ef003
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

34,325,882.558

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
3614552198efe0dae1c66cecb5c7adf6335fb1c1754166542e1b24cb66a49544
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

34,325,882.558

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
9d813fa00272ea895c1e5813457606f472ada60e30cb0277b494f16e57225a85
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

34,199,702.057

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
8015411fbcbd51583c5281fa8cb46100154fff757f1b283a2816d1df030c68e6
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

LLM-Perf

34,199,702.057

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
deci
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
d6ac8ccefaa3b9db179be578f000e44ad572d6e52c29b43e90881ace5dcb5880
complete_configuration_location
source record config and original CSV artifact
source model id
Deci/DeciLM-7B
Original result ↗

Original sources

1–4 / 4

Hugging Face · Deci/DeciLM-7B ↗

Hugging Face · Catalog ↗

Hugging Face · Catalog ↗

Hugging Face · Catalog ↗

Source links

1–3 / 3

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace