Salesforce/codegen-16B-nl

Explore Salesforce/codegen-16B-nl: model specifications, capabilities, benchmark results, platform pricing and original sources.

huggingface/Salesforce/codegen-16B-nl

Specifications & capabilities

Input
text
Output
text
Published model ID
codegen-16B-nl

Evaluations

1–20 / 2,213

LLM-Perf

1,924.85

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
08fe9c3cb1c713be09ad4b00e7d9418f2823de644331b180fdb44e18eba9787a
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,924.85

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
a5f62c30908c74fb94a464a49d09a1e7b0f9f52f40c9142dc0c133cb5d12ed69
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,914.286

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
be446c0f292dcce76dab6d5b2158e37f0c93fbf5a6a948c3a4d6c26d5102ad81
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,914.286

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
a47a81d6c252ab8631fc71e95b9546c23b846086c08f3e48abbd9658b8b1ad0d
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,910.89

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
08fe9c3cb1c713be09ad4b00e7d9418f2823de644331b180fdb44e18eba9787a
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,910.89

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
a5f62c30908c74fb94a464a49d09a1e7b0f9f52f40c9142dc0c133cb5d12ed69
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,899.663

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
be446c0f292dcce76dab6d5b2158e37f0c93fbf5a6a948c3a4d6c26d5102ad81
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,899.663

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
a47a81d6c252ab8631fc71e95b9546c23b846086c08f3e48abbd9658b8b1ad0d
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,892.073

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
733de374baca31d6a65dec20aa0c88440e8ff0307a99b4cfdcefad49356bddcf
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,892.073

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
e972264e8c97105a93f8ef99ce58baddd1145be41b2f7accce3ce7a553ee7ff9
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,885.925

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
93529ff0de92a9e1b561bda2be49faa97085d096243b95c1e666fb364150279a
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,885.925

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
aec1a1ab3558ccca7bbe393da415a45b2b7fc2a7337c64213963446b2139b3fa
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,883.185

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
e972264e8c97105a93f8ef99ce58baddd1145be41b2f7accce3ce7a553ee7ff9
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,883.185

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
733de374baca31d6a65dec20aa0c88440e8ff0307a99b4cfdcefad49356bddcf
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,873.637

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
93529ff0de92a9e1b561bda2be49faa97085d096243b95c1e666fb364150279a
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,873.637

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
aec1a1ab3558ccca7bbe393da415a45b2b7fc2a7337c64213963446b2139b3fa
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,433.993

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
9dd45855ea336381cd44e3075d77a4a8ea78e05a0d7c00cab06e06d30ef7fdcf
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,433.993

source id
llmPerf
leaderboard name
decode memory max_ram
dimension
runtime
metric
decode memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
decode memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
5fff57a4828146ff43ace6228a38aed203e3085fa59fa2ab837d680d838e3d64
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,408.946

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
9dd45855ea336381cd44e3075d77a4a8ea78e05a0d7c00cab06e06d30ef7fdcf
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

1,408.946

source id
llmPerf
leaderboard name
prefill memory max_ram
dimension
runtime
metric
prefill memory max_ram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill memory max_ram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
5fff57a4828146ff43ace6228a38aed203e3085fa59fa2ab837d680d838e3d64
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

Original sources

1–4 / 4

Hugging Face · Model repository ↗

Hugging Face · Catalog ↗

Hugging Face · Catalog ↗

Hugging Face · Catalog ↗

Source links

1–3 / 3

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace