mistralai/Mistral-7B-Instruct-v0.2

Explore mistralai/Mistral-7B-Instruct-v0.2: model specifications, capabilities, benchmark results, platform pricing and original sources.

huggingface/mistralai/Mistral-7B-Instruct-v0.2

Specifications & capabilities

Published model ID
Mistral-7B-Instruct-v0.2

Evaluations

21–40 / 530

LLM-Perf

15,365.308

source id
llmPerf
leaderboard name
prefill memory max_global_vram
dimension
runtime
metric
prefill memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
a8948c265b11bff1341a64b2abfb982e4274e4c3548b37e1435759c8f002f893
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,365.308

source id
llmPerf
leaderboard name
prefill memory max_global_vram
dimension
runtime
metric
prefill memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
bb4c6a4432e2df1327b52a25c0c7ca695a70683ac839151cd5b41a0ec19cf3ae
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,365.308

source id
llmPerf
leaderboard name
prefill memory max_global_vram
dimension
runtime
metric
prefill memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
a535abff071604ee613b61c283dfc76aa42739bd511352b2f62d7ed0ca164a97
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,365.308

source id
llmPerf
leaderboard name
prefill memory max_global_vram
dimension
runtime
metric
prefill memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
c9662d8803eea33761363514d8b4de667e06c745f5789a751e7a553288fb4d45
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,365.308

source id
llmPerf
leaderboard name
decode memory max_global_vram
dimension
runtime
metric
decode memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
c9662d8803eea33761363514d8b4de667e06c745f5789a751e7a553288fb4d45
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,365.308

source id
llmPerf
leaderboard name
decode memory max_global_vram
dimension
runtime
metric
decode memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
a535abff071604ee613b61c283dfc76aa42739bd511352b2f62d7ed0ca164a97
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,365.308

source id
llmPerf
leaderboard name
prefill memory max_global_vram
dimension
runtime
metric
prefill memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
0cfc84341f758a121cc2b27eed4dbe03803e2cc9a4bb8c667701d057b99826a4
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,365.308

source id
llmPerf
leaderboard name
decode memory max_global_vram
dimension
runtime
metric
decode memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
bb4c6a4432e2df1327b52a25c0c7ca695a70683ac839151cd5b41a0ec19cf3ae
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,365.308

source id
llmPerf
leaderboard name
decode memory max_global_vram
dimension
runtime
metric
decode memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
a8948c265b11bff1341a64b2abfb982e4274e4c3548b37e1435759c8f002f893
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,365.308

source id
llmPerf
leaderboard name
decode memory max_global_vram
dimension
runtime
metric
decode memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
8ae9c9fcbd99401a008c2a45db2a18c8c4110dddf5e5f931baf83085862cf5e3
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,256.257

source id
llmPerf
leaderboard name
load_model memory max_global_vram
dimension
runtime
metric
load_model memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
load_model memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
a8948c265b11bff1341a64b2abfb982e4274e4c3548b37e1435759c8f002f893
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,256.257

source id
llmPerf
leaderboard name
load_model memory max_global_vram
dimension
runtime
metric
load_model memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
load_model memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
0cfc84341f758a121cc2b27eed4dbe03803e2cc9a4bb8c667701d057b99826a4
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,256.257

source id
llmPerf
leaderboard name
load_model memory max_global_vram
dimension
runtime
metric
load_model memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
load_model memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
8ae9c9fcbd99401a008c2a45db2a18c8c4110dddf5e5f931baf83085862cf5e3
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,256.257

source id
llmPerf
leaderboard name
load_model memory max_global_vram
dimension
runtime
metric
load_model memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
load_model memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
bb4c6a4432e2df1327b52a25c0c7ca695a70683ac839151cd5b41a0ec19cf3ae
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,256.257

source id
llmPerf
leaderboard name
load_model memory max_global_vram
dimension
runtime
metric
load_model memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
load_model memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
a535abff071604ee613b61c283dfc76aa42739bd511352b2f62d7ed0ca164a97
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

15,256.257

source id
llmPerf
leaderboard name
load_model memory max_global_vram
dimension
runtime
metric
load_model memory max_global_vram
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
load_model memory max_global_vram
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
c9662d8803eea33761363514d8b4de667e06c745f5789a751e7a553288fb4d45
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

14,711.521

source id
llmPerf
leaderboard name
decode memory max_reserved
dimension
runtime
metric
decode memory max_reserved
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_reserved
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
a8948c265b11bff1341a64b2abfb982e4274e4c3548b37e1435759c8f002f893
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

14,711.521

source id
llmPerf
leaderboard name
decode memory max_reserved
dimension
runtime
metric
decode memory max_reserved
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
float16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
flash_attention_2
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_reserved
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
8ae9c9fcbd99401a008c2a45db2a18c8c4110dddf5e5f931baf83085862cf5e3
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

14,711.521

source id
llmPerf
leaderboard name
prefill memory max_reserved
dimension
runtime
metric
prefill memory max_reserved
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
eager
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
prefill memory max_reserved
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
c9662d8803eea33761363514d8b4de667e06c745f5789a751e7a553288fb4d45
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

LLM-Perf

14,711.521

source id
llmPerf
leaderboard name
decode memory max_reserved
dimension
runtime
metric
decode memory max_reserved
rank
1
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
execution
config.backend.name
pytorch
config.backend.seed
42
config.backend.task
text-generation
config.backend.device
cuda
config.backend.library
transformers
config.backend.version
2.5.1+cu124
config.environment.cpu
AMD EPYC 7R32
config.environment.gpu
['NVIDIA A10G']
config.backend.eval_mode
True
config.backend.device_ids
0
config.backend.model_type
mistral
config.backend.no_weights
True
config.environment.system
Linux
config.backend.torch_dtype
bfloat16
config.environment.machine
x86_64
config.backend.torch_compile
False
config.environment.cpu_count
16
config.environment.gpu_count
1
config.environment.processor
x86_64
config.environment.cpu_ram_mb
66697.248768
config.environment.gpu_vram_mb
24146608128
config.backend.autocast_enabled
False
config.environment.peft_version
0.14.0
config.environment.python_version
3.10.12
config.backend.attn_implementation
sdpa
config.backend.deepspeed_inference
False
config.environment.optimum_version
1.23.3
config.backend.to_bettertransformer
False
config.backend.torch_compile_target
forward
config.environment.accelerate_version
1.2.0
config.environment.transformers_version
4.47.0
config.scenario.input_shapes.batch_size
1
config.environment.optimum_benchmark_version
0.5.0.dev0
config.scenario.input_shapes.sequence_length
256
config.backend.model_kwargs.trust_remote_code
True
config.scenario.generate_kwargs.max_new_tokens
64
config.scenario.generate_kwargs.min_new_tokens
64
metric_kind
decode memory max_reserved
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-unquantized-1xA10.csv
published_at
2024-12-13T10:07:03Z
source_revision
2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
metric_population
1
benchmark_evaluation
true
comparison_configuration_sha256
bb4c6a4432e2df1327b52a25c0c7ca695a70683ac839151cd5b41a0ec19cf3ae
complete_configuration_location
source record config and original CSV artifact
source model id
mistralai/Mistral-7B-Instruct-v0.2
Original result ↗

Provider specifications & access

1–5 / 5

mistralai/Mistral-7B-Instruct-v0.2 ↗

source model id
HuggingFace/mistralai-mistral-7b-instruct-v0.2@4
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    mistralai/Mistral-7B-Instruct-v0.2 ↗

    source model id
    HuggingFace/mistralai-mistral-7b-instruct-v0.2@5
    hosting
    foundry-managed-compute
    platforms
    • Microsoft Foundry
    license
    apache-2.0
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities

      mistralai/Mistral-7B-Instruct-v0.2 ↗

      source model id
      HuggingFace/mistralai-mistral-7b-instruct-v0.2@6
      hosting
      foundry-managed-compute
      platforms
      • Microsoft Foundry
      license
      apache-2.0
      architecture
      input_modalities
      • text
      output_modalities
      • text
      capabilities

        mistralai/Mistral-7B-Instruct-v0.2 ↗

        source model id
        publishers/hf-mistralai/models/mistral-7b-instruct-v0.2@001
        hosting
        vertex-self-deployed
        platforms
        • Google Cloud Vertex AI
        architecture
        input_modalities
        • text
        output_modalities
        • text
        capabilities

          Mistral-7B-Instruct-v0.2 ↗

          source model id
          publishers/mistral-ai/models/mistral@mistral-7b-instruct-v0.2
          hosting
          vertex-self-deployed
          platforms
          • Google Cloud Vertex AI
          architecture
          input_modalities
          • text
          output_modalities
          • text
          capabilities

            Safety assessments

            1–1 / 1

            Foundation Model Transparency Index 2025 ↗

            18 / 100

            source id
            stanfordFMTI
            canonical model id
            huggingface/mistralai/Mistral-7B-Instruct-v0.2
            taxonomy
            transparency
            hazard category
            transparency
            hazard name
            Organization transparency
            policy area
            Mistral
            assessment type
            organization-level disclosure
            metric
            disclosure_score
            result label
            18 / 100
            sample size
            100
            updated at
            2026-11-09T18:08:05.227Z

            Original sources

            1–18 / 18

            Hugging Face · Model repository ↗

            Google Cloud · Model page ↗

            Google Cloud · Model page ↗

            Microsoft Foundry · Model page ↗

            Microsoft Foundry · Model page ↗

            Microsoft Foundry · Model page ↗

            Google Cloud · Model documentation ↗

            Google Cloud · Model documentation ↗

            Microsoft Foundry · Model documentation ↗

            arXiv · Paper · 2310.06825 ↗

            GitHub · Catalog ↗

            Google Cloud · Catalog ↗

            Hugging Face · Catalog ↗

            Hugging Face · Catalog ↗

            Hugging Face · Catalog ↗

            Hugging Face · Catalog ↗

            Hugging Face · Catalog ↗

            Microsoft Foundry · Catalog ↗

            Source links

            1–8 / 8

            Hugging Face model card

            1–1 / 1

            Explore more models

            Models

            Loading
            EVIDENCE / COMPARISON

            Benchmarks

            Explore every benchmark

            Loading

            These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

            Install TensorTrace