Qwen/Qwen2-0.5B

Explore Qwen/Qwen2-0.5B: model specifications, capabilities, benchmark results, platform pricing and original sources.

vertexAI/publishers/hf-qwen/models/qwen2-0.5b@001

Specifications & capabilities

Input
text
Output
text
Published model ID
Qwen2-0.5B

Evaluations

1–20 / 1,598

LLM-Perf

240,848,185.827

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
c9623e94498654627a75c767f0d3c17a312fbd258cff4096431fd9d1b57c743d
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

229,633,652.32

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
a3b6b438328dcacd974d8b8c3ef27dbb8d3f2f72269cd104a1e9fa761fe3f828
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

218,135,636.342

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
f084ecfced4abe9646863629dff755122917baed7885784b3547cf2316bb86a5
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

213,553,525.137

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
4be9eb7d97faac69700fdaec04a032b48a745ca6afd29e6f0d7fbece5fa8b775
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

212,786,880.696

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
fbafe61a6c14e2d2f0cab44b0dcdc91e9d467649fe988cc8890e9e9b132a6427
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

211,140,578.203

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
523c745ea2a973da76d0f01d2cba815f45d39d1203258a3d0aad373a7f3e5881
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

209,163,816.203

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
69f6eaf043c02e8037dd82020a6707eba848bd64c02a81d0aaf267087ea48a3e
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

194,906,930.168

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
940a6ce2ac798b7410ea2bf880a3cc595e9896c4fde1564a8e6679c5b4ad1cae
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

189,104,923.812

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
70f81c85983a51383a73ff3dff80097d5c3482d3df0005e17f1da7d5e8870890
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

182,736,957.636

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
a8e15705c82a5c11f763daae098eab233ea1e431111507401c7071c78a44ebef
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

174,290,314.6

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
0ade7ffb30105b1e49a81a1aba40edabbbb1585eb22366b57ec15b23f43c826d
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

171,746,789.007

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
gptq
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.backend.quantization_config.bits
4
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.backend.quantization_config.version
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.model_seqlen
256
config.backend.quantization_config.use_exllama
True
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-gptq-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
ed30d47f4f690c9499ff1f630086077061e56e2271045a9ed3b91a7d788257ad
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

134,578,095.839

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float32
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
16250e0aaf5126670d1b8380eadcabe74606fdf83d3cfe813d5d426ead84b25e
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

128,486,155.723

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float32
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.261056
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
55ca36b781915e362962cdd55eca79ced6f608d1496e3acfe09d691bac2a3cab
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

84,639,794.384

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float32
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
fce81013401834f60eb543be555161cfdf5e3110ba579c86f7ed087f08722b01
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

81,519,486.829

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float32
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
e56b06b9912e4dae8a62dfcbf394d329d6fb3c730321cba09911eb0d77d1bf3a
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

75,247,082.26

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.751424
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
0d105ef9942745e49af5fa5bba8b0d583b2a5b4d102c7a6f5f4a29f3f477ff52
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

73,069,257.107

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.751424
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_4bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
8d81a45a037101732b54eac56d973a91c37b7a50655faf2272fe63bc4949606c
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

61,551,537.87

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.751424
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.backend.quantization_scheme
bnb
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
config.backend.quantization_config.load_in_8bit
True
config.backend.quantization_config.llm_int8_threshold
0.0
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
4750fae75e4f5887e6cd63be94e250e8ef5d139a7e692a505053b96090fc6de2
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

LLM-Perf

61,354,044.054

source id
llmPerf
leaderboard name
prefill efficiency value
dimension
runtime
metric
prefill efficiency value
rank
1
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.0+cu124
config.environment.cpu
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
config.environment.gpu
['Tesla T4']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
qwen2
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
8
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
33163.759616
config.environment.gpu_vram_mb
16106127360
config.backend.autocast_enabled
False
config.environment.peft_version
0.13.2
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.2
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.0.1
config.environment.transformers_version
4.45.2
config.scenario.input_shapes.batch_size
1
config.scenario.input_shapes.num_choices
2
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill efficiency value
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xT4.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
c4c11382c9c1f581b33103e9105d8d761608a11f6aa74aa9913382cc8ca51c0c
complete_configuration_location
source record config and original CSV artifact
source model id
Qwen/Qwen2-0.5B
Original result ↗

Provider specifications & access

1–2 / 2

Qwen/Qwen2-0.5B ↗

source model id
HuggingFace/qwen-qwen2-0.5b@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    Qwen/Qwen2-0.5B ↗

    source model id
    publishers/hf-qwen/models/qwen2-0.5b@001
    hosting
    vertex-self-deployed
    platforms
    • Google Cloud Vertex AI
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities

      Safety assessments

      1–1 / 1

      Foundation Model Transparency Index 2025 ↗

      26 / 100

      source id
      stanfordFMTI
      canonical model id
      vertexAI/publishers/hf-qwen/models/qwen2-0.5b@001
      taxonomy
      transparency
      hazard category
      transparency
      hazard name
      Organization transparency
      policy area
      Alibaba
      assessment type
      organization-level disclosure
      metric
      disclosure_score
      result label
      26 / 100
      sample size
      100
      updated at
      2026-11-09T18:08:05.227Z

      Original sources

      1–12 / 12

      Hugging Face · Model repository ↗

      Google Cloud · Model page ↗

      Microsoft Foundry · Model page ↗

      Google Cloud · Model documentation ↗

      Microsoft Foundry · Model documentation ↗

      GitHub · Catalog ↗

      Google Cloud · Catalog ↗

      Hugging Face · Catalog ↗

      Hugging Face · Catalog ↗

      Hugging Face · Catalog ↗

      Hugging Face · Catalog ↗

      Microsoft Foundry · Catalog ↗

      Source links

      1–7 / 7

      Explore more models

      Models

      Loading
      EVIDENCE / COMPARISON

      Benchmarks

      Explore every benchmark

      Loading

      These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

      Install TensorTrace