OpenAI: GPT-4.1
GPT-4.1 is a flagship large language model optimized for advanced instruction following, real-world software engineering, and long-context reasoning. It supports a 1 million token context window and …
openai/gpt-4.1-2025-04-14
GPT-4.1 is a flagship large language model optimized for advanced instruction following, real-world software engineering, and long-context reasoning. It supports a 1 million token context window and outperforms GPT-4o and GPT-4.5 across coding (54.6% SWE-bench Verified), instruction compliance (87.4% IFEval), and multimodal understanding benchmarks. It is tuned for precise code diffs, agent reliability, and high recall in large document contexts, making it ideal for agents, IDE tooling, and enterprise knowledge retrieval.
Specifications & capabilities
- Input
- image, text, file
- Output
- text
- Context window · tokens
- 1047576
- Published model ID
- gpt-4.1-2025-04-14
- Capabilities
- fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗Evaluations
61–80 / 257
0.883
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite English · English
- dimension
- reasoning
- metric
- Global MMLU Lite English · English
- rank
- 24
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41749
- name
- English
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 153
- name
- Global MMLU Lite English
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 135
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite English
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite English · English
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 135/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-english/versions/1
- measurements
- name
- English
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite English · English
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8825
- standardDeviation
- 0
- confidenceInterval
- 0.0315569037846059
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0315569037846059
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8825
- standardDeviation
- 0
- confidenceInterval
- 0.0315569037846059
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0315569037846059
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41749
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-english
- sort_order
- DESCENDING
- source_name
- English
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:49.982103800Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8509430962153941
- confidence upper
- 0.9140569037846058
Original result ↗0.875
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite German · German
- dimension
- reasoning
- metric
- Global MMLU Lite German · German
- rank
- 24
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41738
- name
- German
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 155
- name
- Global MMLU Lite German
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 137
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite German
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite German · German
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 137/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-german/versions/1
- measurements
- name
- German
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite German · German
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.875
- standardDeviation
- 0
- confidenceInterval
- 0.0324098580108514
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0324098580108514
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.875
- standardDeviation
- 0
- confidenceInterval
- 0.0324098580108514
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0324098580108514
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41738
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-german
- sort_order
- DESCENDING
- source_name
- German
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:45.767915600Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8425901419891486
- confidence upper
- 0.9074098580108514
Original result ↗0.858
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Burmese · Burmese
- dimension
- reasoning
- metric
- Global MMLU Lite Burmese · Burmese
- rank
- 24
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41736
- name
- Burmese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 175
- name
- Global MMLU Lite Burmese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 153
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Burmese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Burmese · Burmese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 153/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-burmese/versions/1
- measurements
- name
- Burmese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Burmese · Burmese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8575
- standardDeviation
- 0
- confidenceInterval
- 0.0342564686873586
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0342564686873586
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8575
- standardDeviation
- 0
- confidenceInterval
- 0.0342564686873586
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0342564686873586
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41736
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-burmese
- sort_order
- DESCENDING
- source_name
- Burmese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:19.694332700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8232435313126414
- confidence upper
- 0.8917564686873587
Original result ↗0.873
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Japanese · Japanese
- dimension
- reasoning
- metric
- Global MMLU Lite Japanese · Japanese
- rank
- 25
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41745
- name
- Japanese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 160
- name
- Global MMLU Lite Japanese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 142
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Japanese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Japanese · Japanese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 142/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-japanese/versions/1
- measurements
- name
- Japanese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Japanese · Japanese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8725
- standardDeviation
- 0
- confidenceInterval
- 0.0326855581520567
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0326855581520567
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8725
- standardDeviation
- 0
- confidenceInterval
- 0.0326855581520567
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0326855581520567
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41745
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-japanese
- sort_order
- DESCENDING
- source_name
- Japanese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:34.896663Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8398144418479434
- confidence upper
- 0.9051855581520567
Original result ↗0.87
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Chinese · Chinese
- dimension
- reasoning
- metric
- Global MMLU Lite Chinese · Chinese
- rank
- 25
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41735
- name
- Chinese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 166
- name
- Global MMLU Lite Chinese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 148
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Chinese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Chinese · Chinese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 148/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-chinese/versions/1
- measurements
- name
- Chinese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Chinese · Chinese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.87
- standardDeviation
- 0
- confidenceInterval
- 0.0329571309666248
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0329571309666248
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.87
- standardDeviation
- 0
- confidenceInterval
- 0.0329571309666248
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0329571309666248
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41735
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-chinese
- sort_order
- DESCENDING
- source_name
- Chinese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:22.953186100Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8370428690333752
- confidence upper
- 0.9029571309666248
Original result ↗0.863
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Bengali · Bengali
- dimension
- reasoning
- metric
- Global MMLU Lite Bengali · Bengali
- rank
- 25
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41751
- name
- Bengali
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 154
- name
- Global MMLU Lite Bengali
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 136
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Bengali
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Bengali · Bengali
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 136/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-bengali/versions/1
- measurements
- name
- Bengali
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Bengali · Bengali
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8625
- standardDeviation
- 0
- confidenceInterval
- 0.0337480742790123
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0337480742790123
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8625
- standardDeviation
- 0
- confidenceInterval
- 0.0337480742790123
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0337480742790123
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41751
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-bengali
- sort_order
- DESCENDING
- source_name
- Bengali
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:47.931282Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8287519257209878
- confidence upper
- 0.8962480742790123
Original result ↗30.928%
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode · SciCode Subproblem Standard
- dimension
- coding
- metric
- SciCode · SciCode Subproblem Standard
- rank
- 25
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40106
- name
- SciCode Subproblem Standard
- version
- 3
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 16
- name
- SciCode
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 16
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode · SciCode Subproblem Standard
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 16/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode/versions/1
- measurements
- name
- SciCode Subproblem Standard
- unit
- percent
- scale
- 1
- metric
- SciCode · SciCode Subproblem Standard
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.3092783505154639
- standardDeviation
- 0
- confidenceInterval
- 0.0531040802020165
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0531040802020165
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.3092783505154639
- standardDeviation
- 0
- confidenceInterval
- 0.0531040802020165
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0531040802020165
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40106
- task_url
- /benchmarks/open-benchmarks/scicode-subproblem-standard/versions/1
- sort_order
- DESCENDING
- source_name
- SciCode Subproblem Standard
- display_type
- percentages
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-21T20:29:26.251536900Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 62
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.25617427031344736
- confidence upper
- 0.3623824307174804
Original result ↗9.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Khmer · EM Khmer
- dimension
- reasoning
- metric
- MultiLoKo Khmer · EM Khmer
- rank
- 25
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 43
- name
- EM Khmer
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 66
- name
- MultiLoKo Khmer
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 52
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Khmer
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Khmer · EM Khmer
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 52/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-khmer/versions/1
- measurements
- name
- EM Khmer
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Khmer · EM Khmer
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.092
- standardDeviation
- 0
- confidenceInterval
- 0.0358273696520135
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0358273696520135
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.092
- standardDeviation
- 0
- confidenceInterval
- 0.0358273696520135
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0358273696520135
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 43
- sort_order
- DESCENDING
- source_name
- EM Khmer
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.0561726303479865
- confidence upper
- 0.1278273696520135
Original result ↗4.615%
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode Main Standard · SciCode Main Standard
- dimension
- coding
- metric
- SciCode Main Standard · SciCode Main Standard
- rank
- 25
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40105
- name
- SciCode Main Standard
- version
- 3
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 12
- name
- SciCode Main Standard
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 12
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode Main Standard
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode Main Standard · SciCode Main Standard
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 12/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode-main-standard/versions/1
- measurements
- name
- SciCode Main Standard
- unit
- percent
- scale
- 1
- metric
- SciCode Main Standard · SciCode Main Standard
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0461538461538461
- standardDeviation
- 0
- confidenceInterval
- 0.051007557541514
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.051007557541514
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0461538461538461
- standardDeviation
- 0
- confidenceInterval
- 0.051007557541514
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.051007557541514
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40105
- task_url
- /benchmarks/tasks/benchmarkler/scicode-standard-scicode-main-standard
- sort_order
- DESCENDING
- source_name
- SciCode Main Standard
- display_type
- percentages
- task_version
- 3
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 57
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- -0.004853711387667901
- confidence upper
- 0.09716140369536011
Original result ↗0.875
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Portuguese · Portuguese
- dimension
- reasoning
- metric
- Global MMLU Lite Portuguese · Portuguese
- rank
- 26
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41746
- name
- Portuguese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 162
- name
- Global MMLU Lite Portuguese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 144
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Portuguese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Portuguese · Portuguese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 144/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-portuguese/versions/1
- measurements
- name
- Portuguese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Portuguese · Portuguese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.875
- standardDeviation
- 0
- confidenceInterval
- 0.0324098580108514
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0324098580108514
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.875
- standardDeviation
- 0
- confidenceInterval
- 0.0324098580108514
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0324098580108514
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41746
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-portuguese
- sort_order
- DESCENDING
- source_name
- Portuguese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:30.662890500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8425901419891486
- confidence upper
- 0.9074098580108514
Original result ↗0.87
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Korean · Korean
- dimension
- reasoning
- metric
- Global MMLU Lite Korean · Korean
- rank
- 27
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41748
- name
- Korean
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 161
- name
- Global MMLU Lite Korean
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 143
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Korean
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Korean · Korean
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 143/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-korean/versions/1
- measurements
- name
- Korean
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Korean · Korean
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.87
- standardDeviation
- 0
- confidenceInterval
- 0.0329571309666248
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0329571309666248
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.87
- standardDeviation
- 0
- confidenceInterval
- 0.0329571309666248
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0329571309666248
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41748
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-korean
- sort_order
- DESCENDING
- source_name
- Korean
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:33.197926100Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8370428690333752
- confidence upper
- 0.9029571309666248
Original result ↗86.52%
- source id
- bfcl
- leaderboard name
- BFCL Irrelevance Detection
- dimension
- toolUse
- metric
- Irrelevance detection
- rank
- 27
Original result ↗0.092
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode Main With Background · SciCode Main with Background
- dimension
- coding
- metric
- SciCode Main With Background · SciCode Main with Background
- rank
- 27
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40201
- name
- SciCode Main with Background
- version
- 3
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 13
- name
- SciCode Main With Background
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 13
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode Main With Background
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode Main With Background · SciCode Main with Background
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 13/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode-main-with-background/versions/1
- measurements
- name
- SciCode Main with Background
- unit
- score
- scale
- 1
- metric
- SciCode Main With Background · SciCode Main with Background
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0923076923076923
- standardDeviation
- 0
- confidenceInterval
- 0.0703687257914088
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0703687257914088
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0923076923076923
- standardDeviation
- 0
- confidenceInterval
- 0.0703687257914088
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0703687257914088
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40201
- task_url
- /benchmarks/tasks/benchmarkler/scicode-background-scicode-main-with-background
- sort_order
- DESCENDING
- source_name
- SciCode Main with Background
- display_type
- rawValues
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 57
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.021938966516283492
- confidence upper
- 0.1626764180991011
Original result ↗87.5%
- source id
- bfcl
- leaderboard name
- BFCL Relevance Detection
- dimension
- toolUse
- metric
- Relevance detection
- rank
- 28
Original result ↗90.315%
- source id
- kaggleBenchmarks
- leaderboard name
- MMLU · mmlu_task
- dimension
- reasoning
- metric
- MMLU · mmlu_task
- rank
- 29
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40139
- name
- mmlu_task
- version
- 7
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 42
- name
- MMLU
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- version_id
- 28
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MMLU
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MMLU · mmlu_task
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 28/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mmlu/versions/1
- measurements
- name
- mmlu_task
- unit
- percent
- scale
- 1
- metric
- MMLU · mmlu_task
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9031476997578692
- standardDeviation
- 0
- confidenceInterval
- 0.004891791821091
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.004891791821091
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9031476997578692
- standardDeviation
- 0
- confidenceInterval
- 0.004891791821091
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.004891791821091
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40139
- task_url
- /benchmarks/tasks/benchmarkler/mmlu-task
- sort_order
- DESCENDING
- source_name
- mmlu_task
- display_type
- percentages
- task_version
- 7
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 61
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-30T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8982559079367782
- confidence upper
- 0.9080394915789602
Original result ↗38.88%
- source id
- bfcl
- leaderboard name
- BFCL Multi Turn
- dimension
- agentic
- metric
- Multi-turn accuracy
- rank
- 29
Original result ↗0.381
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode Subproblem With Background · SciCode Subproblem with Background
- dimension
- coding
- metric
- SciCode Subproblem With Background · SciCode Subproblem with Background
- rank
- 29
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40200
- name
- SciCode Subproblem with Background
- version
- 3
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 15
- name
- SciCode Subproblem With Background
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 15
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode Subproblem With Background
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode Subproblem With Background · SciCode Subproblem with Background
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 15/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode-subproblem-with-background/versions/1
- measurements
- name
- SciCode Subproblem with Background
- unit
- score
- scale
- 1
- metric
- SciCode Subproblem With Background · SciCode Subproblem with Background
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.3814432989690721
- standardDeviation
- 0
- confidenceInterval
- 0.0558092709790888
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0558092709790888
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.3814432989690721
- standardDeviation
- 0
- confidenceInterval
- 0.0558092709790888
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0558092709790888
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40200
- task_url
- /benchmarks/tasks/benchmarkler/scicode-background-scicode-subproblem-with-background
- sort_order
- DESCENDING
- source_name
- SciCode Subproblem with Background
- display_type
- rawValues
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 57
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.3256340279899833
- confidence upper
- 0.43725256994816086
Original result ↗4.615%
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode · SciCode Main Standard
- dimension
- coding
- metric
- SciCode · SciCode Main Standard
- rank
- 29
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40105
- name
- SciCode Main Standard
- version
- 3
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 16
- name
- SciCode
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 16
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode · SciCode Main Standard
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 16/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode/versions/1
- measurements
- name
- SciCode Main Standard
- unit
- percent
- scale
- 1
- metric
- SciCode · SciCode Main Standard
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0461538461538461
- standardDeviation
- 0
- confidenceInterval
- 0.051007557541514
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.051007557541514
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0461538461538461
- standardDeviation
- 0
- confidenceInterval
- 0.051007557541514
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.051007557541514
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40105
- task_url
- /benchmarks/open-benchmarks/scicode-main-standard/versions/1
- sort_order
- DESCENDING
- source_name
- SciCode Main Standard
- display_type
- percentages
- task_version
- 3
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-10-21T20:29:26.251536900Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 62
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- -0.004853711387667901
- confidence upper
- 0.09716140369536011
Original result ↗9.231%
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode · SciCode Main With Background
- dimension
- coding
- metric
- SciCode · SciCode Main With Background
- rank
- 31
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40201
- name
- SciCode Main With Background
- version
- 3
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 16
- name
- SciCode
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 16
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode · SciCode Main With Background
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 16/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode/versions/1
- measurements
- name
- SciCode Main With Background
- unit
- percent
- scale
- 1
- metric
- SciCode · SciCode Main With Background
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0923076923076923
- standardDeviation
- 0
- confidenceInterval
- 0.0703687257914088
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0703687257914088
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0923076923076923
- standardDeviation
- 0
- confidenceInterval
- 0.0703687257914088
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0703687257914088
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40201
- task_url
- /benchmarks/open-benchmarks/scicode-main-with-background/versions/1
- sort_order
- DESCENDING
- source_name
- SciCode Main With Background
- display_type
- percentages
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-21T20:29:26.251536900Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 62
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.021938966516283492
- confidence upper
- 0.1626764180991011
Original result ↗35%
- source id
- bfcl
- leaderboard name
- BFCL Web Search
- dimension
- agentic
- metric
- Web-search accuracy
- rank
- 32
Original result ↗Published prices
1–20 / 36
$0.40
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Cached input tokens (per 1M)
$0.00
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- cache_write_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Cache write, 5 minutes (per 1M)
$1.61
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Input tokens (per 1M)
$6.46
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Output tokens (per 1M)
$0.50
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$2.00
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$8.00
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$2.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$8.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
$0.25
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$1.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$4.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
- Billing conditions
- mode
- batch
$0.50
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$2.00
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$8.00
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- request
- Currency
- USD
- Billing conditions
- charge
- web_search
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Web search
- Pricing via
- Vercel AI Gateway
$0.50
- Inference provider
- vercelAIGatewayCatalog
- Platform
- openai
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- openai | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–4 / 4
- source model id
- azure-openai/gpt-4.1@2025-04-14
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 1048576
- max output tokens
- 32768
- source model id
- gpt-4.1
- hosting
- gateway
- context length
- 1047576
- max output tokens
- 32768
- capabilities
- reasoning
- structuredOutputs
- toolCalling
- visionInput
- webSearch
- source model id
- openai/gpt-4.1
- hosting
- third-party
- context length
- 1047576
- max output tokens
- 32768
- source model id
- openai/gpt-4.1
- hosting
- vercel-gateway
- context length
- 1047576
- max output tokens
- 32768
- capabilities
- promptCaching
- structuredOutputs
- toolCalling
- visionInput
- webSearch
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/gpt-4.1-2025-04-14
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–14 / 14
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-30
- normalized value
- 2024-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-30
- normalized value
- 2024-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-01
- normalized value
- 2024-06-01
- precision
- day
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- May 2024
- normalized value
- 2024-05
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models