LLM-Perf
0
- source id
- llmPerf
- leaderboard name
- prefill memory max_process_vram
- dimension
- runtime
- metric
- prefill memory max_process_vram
- rank
- 1
- aggregation scope
- status
- published
- direction
- lowerIsBetter
- conditions
- suite
- optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xA10.csv
- execution
- config.backend.name
- pytorch
- config.backend.seed
- 42
- config.backend.task
- text-generation
- config.backend.device
- cuda
- config.backend.library
- transformers
- config.backend.version
- 2.3.0+cu121
- config.environment.cpu
- AMD EPYC 7R32
- config.environment.gpu
- ['NVIDIA A10G']
- config.backend.eval_mode
- True
- config.backend.device_ids
- 0
- config.backend.no_weights
- True
- config.environment.system
- Linux
- config.backend.torch_dtype
- float16
- config.environment.machine
- x86_64
- config.backend.torch_compile
- False
- config.environment.cpu_count
- 16
- config.environment.gpu_count
- 1
- config.environment.processor
- x86_64
- config.environment.cpu_ram_mb
- 66697.29792
- config.environment.gpu_vram_mb
- 24146608128
- config.backend.autocast_enabled
- False
- config.environment.peft_version
- 0.10.0
- config.environment.python_version
- 3.10.12
- config.backend.attn_implementation
- eager
- config.backend.deepspeed_inference
- False
- config.backend.quantization_scheme
- bnb
- config.environment.optimum_version
- 1.19.2
- config.backend.to_bettertransformer
- False
- config.backend.torch_compile_target
- forward
- config.environment.accelerate_version
- 0.30.1
- config.environment.transformers_version
- 4.40.2
- config.scenario.input_shapes.batch_size
- 1
- config.backend.hub_kwargs.force_download
- False
- config.scenario.input_shapes.num_choices
- 2
- config.backend.hub_kwargs.local_files_only
- False
- config.backend.hub_kwargs.trust_remote_code
- True
- config.environment.optimum_benchmark_version
- 0.2.0
- config.scenario.input_shapes.sequence_length
- 256
- config.scenario.generate_kwargs.max_new_tokens
- 64
- config.scenario.generate_kwargs.min_new_tokens
- 64
- config.backend.quantization_config.load_in_8bit
- True
- config.backend.quantization_config.llm_int8_threshold
- 0.0
- metric_kind
- prefill memory max_process_vram
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- optimum-benchmark/llm-perf-leaderboard/perf-df-pytorch-cuda-bnb-1xA10.csv
- published_at
- 2024-12-13T10:07:03Z
- model_revision
- main
- source_revision
- 2fa8d8c300c4eb6e744d5e35999db926c0c26e90e767e4c38da326eebe21a92b
- metric_population
- 27
- benchmark_evaluation
- true
- comparison_configuration_sha256
- d8d39f453f0d4085b264c536b33ab448035c6b27495933e3e244418b3ca7c5fe
- complete_configuration_location
- source record config and original CSV artifact
- source model id
- Salesforce/codegen-16B-nl