Salesforce/codegen-16B-nl

Explore Salesforce/codegen-16B-nl: model specifications, capabilities, benchmark results, platform pricing and original sources.

huggingface/Salesforce/codegen-16B-nl

Specifications & capabilities

Input
text
Output
text
Published model ID
codegen-16B-nl

Evaluations

161–180 / 2,213

LLM-Perf

0

source id
llmPerf
leaderboard name
decode memory max_process_vram
dimension
runtime
metric
decode memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.3.0+cu121
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.29792
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.10.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.19.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.30.1
config.environment.transformers_version
4.40.2
config.scenario.input_shapes.batch_size
1
config.backend.hub_kwargs.force_download
False
config.scenario.input_shapes.num_choices
2
config.backend.hub_kwargs.local_files_only
False
config.backend.hub_kwargs.trust_remote_code
True
config.environment.optimum_benchmark_version
0.2.0
config.scenario.input_shapes.sequence_length
256
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
decode memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xA10.csv
published_at
2024-12-13T10:07:03Z
model_revision
main
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
28
benchmark_evaluation
true
comparison_configuration_sha256
ea767bd2975aa7d7983a8f02f978e6163d661d877ea0c26759be75c178ad0a3b
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
load memory max_process_vram
dimension
runtime
metric
load memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
load memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
9dd45855ea336381cd44e3075d77a4a8ea78e05a0d7c00cab06e06d30ef7fdcf
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
decode memory max_process_vram
dimension
runtime
metric
decode memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.0
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemm
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
9c186d8748545745cece8fbdd659adc7146ecf98581f79b9614b93b59b722838
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
733de374baca31d6a65dec20aa0c88440e8ff0307a99b4cfdcefad49356bddcf
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
e004ec218915e3d8448057784260ea6fbf81b4ecb9345717a1a8e2db82b2d420
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
decode memory max_process_vram
dimension
runtime
metric
decode memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
decode memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
aec1a1ab3558ccca7bbe393da415a45b2b7fc2a7337c64213963446b2139b3fa
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.0
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
2
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
df897434b6212054034a70d442e1d4d4bb8d2b798a720c8fefded31f90aa2d0b
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
9dd45855ea336381cd44e3075d77a4a8ea78e05a0d7c00cab06e06d30ef7fdcf
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
prefill memory max_process_vram
dimension
runtime
metric
prefill memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
1
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
prefill memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
021b59af319defeebc03332cf10b133a56326ff953a5c5362866e21626566f39
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
load energy gpu
dimension
runtime
metric
load energy gpu
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cpu-unquantized-32vCPU-C7i.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cpu
config.backend.library
transformers
config.backend.version
2.4.1+cpu
config.environment.cpu
Intel(R) Xeon(R) Platinum 8488C
config.backend.eval_mode
True
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
32
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66326.188032
config.backend.autocast_enabled
False
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
load energy gpu
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cpu-unquantized-32vCPU-C7i.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
5932062e2aff2f27394ef3ea79b4909bf366a8554726642b8595bf065564dba0
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
load memory max_process_vram
dimension
runtime
metric
load memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
load memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
08fe9c3cb1c713be09ad4b00e7d9418f2823de644331b180fdb44e18eba9787a
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

0

source id
llmPerf
leaderboard name
load memory max_process_vram
dimension
runtime
metric
load memory max_process_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemv
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
load memory max_process_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
d57eedc9c14587560aa174e08a23823afb8758d28b2a6c450bfb4ea5fb951dc3
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

14,985,937.387

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
2
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
2
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
c92ba93633b0def6411397b11684376e11cbf6f437e52c55bc3159faaf4b13ba
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

14,985,937.387

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
2
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
exllama
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.exllama_config.version
2
config.backend.quantization_config.exllama_config.max_input_len
64
config.backend.quantization_config.exllama_config.max_batch_size
1
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
1df88452f37ef8c74b1fcb7fefe5148087869cf4cde69c894c5031f2cb1e49a6
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

14,885,523.97

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
2
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
be446c0f292dcce76dab6d5b2158e37f0c93fbf5a6a948c3a4d6c26d5102ad81
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

14,885,523.97

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
2
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
a47a81d6c252ab8631fc71e95b9546c23b846086c08f3e48abbd9658b8b1ad0d
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

14,735,272.292

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
2
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
a5f62c30908c74fb94a464a49d09a1e7b0f9f52f40c9142dc0c133cb5d12ed69
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

14,735,272.292

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
2
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.22.0
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-gptq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
08fe9c3cb1c713be09ad4b00e7d9418f2823de644331b180fdb44e18eba9787a
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

11,490,423.752

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
2
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemv
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
ab3619e204b8bf9907ed39df6132c3d1632f5263afad8a9f0846610dae92d3e8
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

LLM-Perf

11,490,423.752

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
2
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.4.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
codegen
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.12.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
awq
config.environment.optimum_version
1.21.4
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
0.34.2
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.44.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
gemv
config.environment.optimum_benchmark_version
0.4.0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-awq-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
2
benchmark_evaluation
true
comparison_configuration_sha256
d57eedc9c14587560aa174e08a23823afb8758d28b2a6c450bfb4ea5fb951dc3
complete_configuration_location
source record config and original CSV artifact
source model id
Salesforce/codegen-16B-nl
Original result ↗

Original sources

1–4 / 4

Hugging Face · Model repository ↗

Hugging Face · Catalog ↗

Hugging Face · Catalog ↗

Hugging Face · Catalog ↗

Source links

1–3 / 3

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace