OpenAI: GPT-4o-mini (2024-07-18)
GPT-4o mini is OpenAI's newest model after [GPT-4 Omni](/models/openai/gpt-4o), supporting both text and image inputs with text outputs. As their most advanced small model, it is many multiples more …
openai/gpt-4o-mini-2024-07-18
GPT-4o mini is OpenAI's newest model after [GPT-4 Omni](/models/openai/gpt-4o), supporting both text and image inputs with text outputs. As their most advanced small model, it is many multiples more affordable...
Specifications & capabilities
- Input
- text, image, file
- Output
- text
- Context window · tokens
- 128000
- Capabilities
- audioInput, fileInput, multimodal, promptCaching, structuredOutputs, toolCalling, visionInput
Evaluations
41–60 / 186
20%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Italian · EM Italian
- dimension
- reasoning
- metric
- MultiLoKo Italian · EM Italian
- rank
- 36
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41
- name
- EM Italian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 64
- name
- MultiLoKo Italian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 50
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Italian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Italian · EM Italian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 50/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-italian/versions/1
- measurements
- name
- EM Italian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Italian · EM Italian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.2
- standardDeviation
- 0
- confidenceInterval
- 0.0495836025843649
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0495836025843649
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.2
- standardDeviation
- 0
- confidenceInterval
- 0.0495836025843649
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0495836025843649
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41
- sort_order
- DESCENDING
- source_name
- EM Italian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.15041639741563512
- confidence upper
- 0.2495836025843649
Original result ↗18.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Hindi · EM Hindi
- dimension
- reasoning
- metric
- MultiLoKo Hindi · EM Hindi
- rank
- 36
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 39
- name
- EM Hindi
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 62
- name
- MultiLoKo Hindi
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 48
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Hindi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Hindi · EM Hindi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 48/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-hindi/versions/1
- measurements
- name
- EM Hindi
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Hindi · EM Hindi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.184
- standardDeviation
- 0
- confidenceInterval
- 0.0480321556116024
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0480321556116024
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.184
- standardDeviation
- 0
- confidenceInterval
- 0.0480321556116024
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0480321556116024
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 39
- sort_order
- DESCENDING
- source_name
- EM Hindi
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.1359678443883976
- confidence upper
- 0.2320321556116024
Original result ↗12.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Vietnamese · EM Vietnamese
- dimension
- reasoning
- metric
- MultiLoKo Vietnamese · EM Vietnamese
- rank
- 36
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 59
- name
- EM Vietnamese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 82
- name
- MultiLoKo Vietnamese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 68
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Vietnamese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Vietnamese · EM Vietnamese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 68/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-vietnamese/versions/1
- measurements
- name
- EM Vietnamese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Vietnamese · EM Vietnamese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.128
- standardDeviation
- 0
- confidenceInterval
- 0.0414134407037423
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0414134407037423
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.128
- standardDeviation
- 0
- confidenceInterval
- 0.0414134407037423
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0414134407037423
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 59
- sort_order
- DESCENDING
- source_name
- EM Vietnamese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.08658655929625769
- confidence upper
- 0.1694134407037423
Original result ↗10.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Korean · EM Korean
- dimension
- reasoning
- metric
- MultiLoKo Korean · EM Korean
- rank
- 36
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 44
- name
- EM Korean
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 67
- name
- MultiLoKo Korean
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 53
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Korean
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Korean · EM Korean
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 53/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-korean/versions/1
- measurements
- name
- EM Korean
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Korean · EM Korean
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.104
- standardDeviation
- 0
- confidenceInterval
- 0.0378397938021634
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0378397938021634
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.104
- standardDeviation
- 0
- confidenceInterval
- 0.0378397938021634
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0378397938021634
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 44
- sort_order
- DESCENDING
- source_name
- EM Korean
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.0661602061978366
- confidence upper
- 0.14183979380216338
Original result ↗8.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Cantonese · EM Cantonese
- dimension
- reasoning
- metric
- MultiLoKo Cantonese · EM Cantonese
- rank
- 36
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 32
- name
- EM Cantonese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 55
- name
- MultiLoKo Cantonese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 41
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Cantonese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Cantonese · EM Cantonese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 41/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-cantonese/versions/1
- measurements
- name
- EM Cantonese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Cantonese · EM Cantonese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.084
- standardDeviation
- 0
- confidenceInterval
- 0.0343847179717669
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0343847179717669
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.084
- standardDeviation
- 0
- confidenceInterval
- 0.0343847179717669
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0343847179717669
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 32
- sort_order
- DESCENDING
- source_name
- EM Cantonese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.04961528202823311
- confidence upper
- 0.1183847179717669
Original result ↗1.538%
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode Main Standard · SciCode Main Standard
- dimension
- coding
- metric
- SciCode Main Standard · SciCode Main Standard
- rank
- 36
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40105
- name
- SciCode Main Standard
- version
- 3
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 12
- name
- SciCode Main Standard
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 12
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode Main Standard
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode Main Standard · SciCode Main Standard
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 12/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode-main-standard/versions/1
- measurements
- name
- SciCode Main Standard
- unit
- percent
- scale
- 1
- metric
- SciCode Main Standard · SciCode Main Standard
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0153846153846153
- standardDeviation
- 0
- confidenceInterval
- 0.0299204446309948
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0299204446309948
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0153846153846153
- standardDeviation
- 0
- confidenceInterval
- 0.0299204446309948
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0299204446309948
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40105
- task_url
- /benchmarks/tasks/benchmarkler/scicode-standard-scicode-main-standard
- sort_order
- DESCENDING
- source_name
- SciCode Main Standard
- display_type
- percentages
- task_version
- 3
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 57
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- -0.014535829246379499
- confidence upper
- 0.0453050600156101
Original result ↗4.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Farsi · EM Farsi
- dimension
- reasoning
- metric
- MultiLoKo Farsi · EM Farsi
- rank
- 37
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 35
- name
- EM Farsi
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 58
- name
- MultiLoKo Farsi
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 44
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Farsi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Farsi · EM Farsi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 44/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-farsi/versions/1
- measurements
- name
- EM Farsi
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Farsi · EM Farsi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.048
- standardDeviation
- 0
- confidenceInterval
- 0.0264982491701198
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0264982491701198
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.048
- standardDeviation
- 0
- confidenceInterval
- 0.0264982491701198
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0264982491701198
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 35
- sort_order
- DESCENDING
- source_name
- EM Farsi
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 50
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.0215017508298802
- confidence upper
- 0.0744982491701198
Original result ↗4.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Khmer · EM Khmer
- dimension
- reasoning
- metric
- MultiLoKo Khmer · EM Khmer
- rank
- 37
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 43
- name
- EM Khmer
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 66
- name
- MultiLoKo Khmer
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 52
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Khmer
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Khmer · EM Khmer
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 52/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-khmer/versions/1
- measurements
- name
- EM Khmer
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Khmer · EM Khmer
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.048
- standardDeviation
- 0
- confidenceInterval
- 0.0264982491701198
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0264982491701198
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.048
- standardDeviation
- 0
- confidenceInterval
- 0.0264982491701198
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0264982491701198
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 43
- sort_order
- DESCENDING
- source_name
- EM Khmer
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.0215017508298802
- confidence upper
- 0.0744982491701198
Original result ↗19.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Tagalog · EM Tagalog
- dimension
- reasoning
- metric
- MultiLoKo Tagalog · EM Tagalog
- rank
- 38
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 54
- name
- EM Tagalog
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 77
- name
- MultiLoKo Tagalog
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 63
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Tagalog
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Tagalog · EM Tagalog
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 63/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-tagalog/versions/1
- measurements
- name
- EM Tagalog
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Tagalog · EM Tagalog
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.196
- standardDeviation
- 0
- confidenceInterval
- 0.0492078224188091
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0492078224188091
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.196
- standardDeviation
- 0
- confidenceInterval
- 0.0492078224188091
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0492078224188091
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 54
- sort_order
- DESCENDING
- source_name
- EM Tagalog
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.1467921775811909
- confidence upper
- 0.24520782241880912
Original result ↗16.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- dimension
- reasoning
- metric
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- rank
- 38
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 56
- name
- EM Traditional Mandarin
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 79
- name
- MultiLoKo Traditional Mandarin
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 65
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Traditional Mandarin
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 65/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-traditional-mandarin/versions/1
- measurements
- name
- EM Traditional Mandarin
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.164
- standardDeviation
- 0
- confidenceInterval
- 0.0458989921524404
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0458989921524404
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.164
- standardDeviation
- 0
- confidenceInterval
- 0.0458989921524404
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0458989921524404
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 56
- sort_order
- DESCENDING
- source_name
- EM Traditional Mandarin
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.11810100784755961
- confidence upper
- 0.2098989921524404
Original result ↗79.115%
- source id
- helm
- leaderboard name
- HELM WildBench
- dimension
- agentic
- metric
- wildbench_score_rescaled
- rank
- 39
- sample size
- 1000
- source model id
- openai/gpt-4o-mini-2024-07-18
Original result ↗17.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo German · EM German
- dimension
- reasoning
- metric
- MultiLoKo German · EM German
- rank
- 39
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 37
- name
- EM German
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 60
- name
- MultiLoKo German
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 46
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo German
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo German · EM German
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 46/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-german/versions/1
- measurements
- name
- EM German
- unit
- percent
- scale
- 1
- metric
- MultiLoKo German · EM German
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.172
- standardDeviation
- 0
- confidenceInterval
- 0.0467797049758963
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0467797049758963
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.172
- standardDeviation
- 0
- confidenceInterval
- 0.0467797049758963
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0467797049758963
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 37
- sort_order
- DESCENDING
- source_name
- EM German
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.1252202950241037
- confidence upper
- 0.21877970497589627
Original result ↗1.538%
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode · SciCode Main Standard
- dimension
- coding
- metric
- SciCode · SciCode Main Standard
- rank
- 41
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40105
- name
- SciCode Main Standard
- version
- 3
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 16
- name
- SciCode
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 16
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode · SciCode Main Standard
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 16/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode/versions/1
- measurements
- name
- SciCode Main Standard
- unit
- percent
- scale
- 1
- metric
- SciCode · SciCode Main Standard
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0153846153846153
- standardDeviation
- 0
- confidenceInterval
- 0.0299204446309948
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0299204446309948
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0153846153846153
- standardDeviation
- 0
- confidenceInterval
- 0.0299204446309948
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0299204446309948
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40105
- task_url
- /benchmarks/open-benchmarks/scicode-main-standard/versions/1
- sort_order
- DESCENDING
- source_name
- SciCode Main Standard
- display_type
- percentages
- task_version
- 3
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-10-21T20:29:26.251536900Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 62
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- -0.014535829246379499
- confidence upper
- 0.0453050600156101
Original result ↗60.3%
- source id
- helm
- leaderboard name
- HELM MMLU-Pro
- dimension
- intelligence
- metric
- chain_of_thought_correctness
- rank
- 47
- sample size
- 1000
- source model id
- openai/gpt-4o-mini-2024-07-18
Original result ↗28%
- source id
- helm
- leaderboard name
- HELM Omni-MATH
- dimension
- intelligence
- metric
- omni_math_accuracy
- rank
- 48
- sample size
- 1000
- source model id
- openai/gpt-4o-mini-2024-07-18
Original result ↗0.165
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode Subproblem Standard · SciCode Subproblem Standard
- dimension
- coding
- metric
- SciCode Subproblem Standard · SciCode Subproblem Standard
- rank
- 48
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40106
- name
- SciCode Subproblem Standard
- version
- 3
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 14
- name
- SciCode Subproblem Standard
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 14
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode Subproblem Standard
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode Subproblem Standard · SciCode Subproblem Standard
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 14/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode-subproblem-standard/versions/1
- measurements
- name
- SciCode Subproblem Standard
- unit
- score
- scale
- 1
- metric
- SciCode Subproblem Standard · SciCode Subproblem Standard
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.1649484536082474
- standardDeviation
- 0
- confidenceInterval
- 0.0426414891568138
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0426414891568138
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.1649484536082474
- standardDeviation
- 0
- confidenceInterval
- 0.0426414891568138
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0426414891568138
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40106
- task_url
- /benchmarks/tasks/benchmarkler/scicode-standard-scicode-subproblem-standard
- sort_order
- DESCENDING
- source_name
- SciCode Subproblem Standard
- display_type
- rawValues
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 57
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.12230696445143359
- confidence upper
- 0.2075899427650612
Original result ↗0.241
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode Subproblem With Background · SciCode Subproblem with Background
- dimension
- coding
- metric
- SciCode Subproblem With Background · SciCode Subproblem with Background
- rank
- 49
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40200
- name
- SciCode Subproblem with Background
- version
- 3
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 15
- name
- SciCode Subproblem With Background
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 15
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode Subproblem With Background
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode Subproblem With Background · SciCode Subproblem with Background
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 15/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode-subproblem-with-background/versions/1
- measurements
- name
- SciCode Subproblem with Background
- unit
- score
- scale
- 1
- metric
- SciCode Subproblem With Background · SciCode Subproblem with Background
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.2405498281786941
- standardDeviation
- 0
- confidenceInterval
- 0.0491081715605116
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0491081715605116
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.2405498281786941
- standardDeviation
- 0
- confidenceInterval
- 0.0491081715605116
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0491081715605116
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40200
- task_url
- /benchmarks/tasks/benchmarkler/scicode-background-scicode-subproblem-with-background
- sort_order
- DESCENDING
- source_name
- SciCode Subproblem with Background
- display_type
- rawValues
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 57
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.19144165661818252
- confidence upper
- 0.2896579997392057
Original result ↗9.063%
- source id
- kaggleBenchmarks
- leaderboard name
- SimpleQA Verified · simpleqa_verified_task
- dimension
- knowledge
- metric
- SimpleQA Verified · simpleqa_verified_task
- rank
- 49
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40086
- name
- simpleqa_verified_task
- version
- 7
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 137
- name
- SimpleQA Verified
- type
- INDIVIDUAL
- owner
- deepmind
- version
- 1
- version_id
- 123
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · deepmind · SimpleQA Verified
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SimpleQA Verified · simpleqa_verified_task
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 123/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/deepmind/simpleqa-verified/versions/1
- measurements
- name
- simpleqa_verified_task
- unit
- percent
- scale
- 1
- metric
- SimpleQA Verified · simpleqa_verified_task
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0906344410876133
- standardDeviation
- 0
- confidenceInterval
- 0.0188596516765591
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0188596516765591
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-07-24T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0906344410876133
- standardDeviation
- 0
- confidenceInterval
- 0.0188596516765591
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0188596516765591
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40086
- task_url
- /benchmarks/tasks/benchmarkler/simpleqa-verified-task
- sort_order
- DESCENDING
- source_name
- simpleqa_verified_task
- display_type
- percentages
- task_version
- 7
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-09-10T15:04:02.690199100Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 57
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-07-24T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.0717747894110542
- confidence upper
- 0.10949409276417241
Original result ↗0
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode Main With Background · SciCode Main with Background
- dimension
- coding
- metric
- SciCode Main With Background · SciCode Main with Background
- rank
- 49
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40201
- name
- SciCode Main with Background
- version
- 3
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 13
- name
- SciCode Main With Background
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 13
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode Main With Background
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode Main With Background · SciCode Main with Background
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 13/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode-main-with-background/versions/1
- measurements
- name
- SciCode Main with Background
- unit
- score
- scale
- 1
- metric
- SciCode Main With Background · SciCode Main with Background
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40201
- task_url
- /benchmarks/tasks/benchmarkler/scicode-background-scicode-main-with-background
- sort_order
- DESCENDING
- source_name
- SciCode Main with Background
- display_type
- rawValues
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 57
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0
- confidence upper
- 0
Original result ↗81.377%
- source id
- kaggleBenchmarks
- leaderboard name
- MMLU · mmlu_task
- dimension
- reasoning
- metric
- MMLU · mmlu_task
- rank
- 50
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40139
- name
- mmlu_task
- version
- 7
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 42
- name
- MMLU
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- version_id
- 28
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MMLU
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MMLU · mmlu_task
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 28/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mmlu/versions/1
- measurements
- name
- mmlu_task
- unit
- percent
- scale
- 1
- metric
- MMLU · mmlu_task
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8137729668138441
- standardDeviation
- 0
- confidenceInterval
- 0.0064388246537501
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0064388246537501
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-05T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8137729668138441
- standardDeviation
- 0
- confidenceInterval
- 0.0064388246537501
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0064388246537501
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40139
- task_url
- /benchmarks/tasks/benchmarkler/mmlu-task
- sort_order
- DESCENDING
- source_name
- mmlu_task
- display_type
- percentages
- task_version
- 7
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-mini-2024-07-18
- metric_population
- 61
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-05T00:00:00.000Z
- source model id
- GPT-4o mini
- confidence lower
- 0.807334142160094
- confidence upper
- 0.8202117914675943
Original result ↗Published prices
1–3 / 3
$0.08
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.15
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$0.60
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
Provider specifications & access
1–1 / 1
- source model id
- azure-openai/gpt-4o-mini@2024-07-18
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 128000
- max input tokens
- 128000
- max output tokens
- 16384
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/gpt-4o-mini-2024-07-18
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–7 / 7
- field
- context_length
- value
- 128000
- normalized value
- 128000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 128000
- normalized value
- 128000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2023-10-31
- normalized value
- 2023-10-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_input_tokens
- value
- 128000
- normalized value
- 128000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 16384
- normalized value
- 16384
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 16384
- normalized value
- 16384
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- September 2023
- normalized value
- 2023-09
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models