OpenAI: GPT-4.1
GPT-4.1 is a flagship large language model optimized for advanced instruction following, real-world software engineering, and long-context reasoning. It supports a 1 million token context window and …
openai/gpt-4.1-2025-04-14
GPT-4.1 is a flagship large language model optimized for advanced instruction following, real-world software engineering, and long-context reasoning. It supports a 1 million token context window and outperforms GPT-4o and GPT-4.5 across coding (54.6% SWE-bench Verified), instruction compliance (87.4% IFEval), and multimodal understanding benchmarks. It is tuned for precise code diffs, agent reliability, and high recall in large document contexts, making it ideal for agents, IDE tooling, and enterprise knowledge retrieval.
Specifications & capabilities
- Input
- image, text, file
- Output
- text
- Context window · tokens
- 1047576
- Published model ID
- gpt-4.1-2025-04-14
- Capabilities
- fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗Evaluations
41–60 / 257
27.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Marathi · EM Marathi
- dimension
- reasoning
- metric
- MultiLoKo Marathi · EM Marathi
- rank
- 16
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 46
- name
- EM Marathi
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 69
- name
- MultiLoKo Marathi
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 55
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Marathi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Marathi · EM Marathi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 55/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-marathi/versions/1
- measurements
- name
- EM Marathi
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Marathi · EM Marathi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.276
- standardDeviation
- 0
- confidenceInterval
- 0.0554117556980965
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0554117556980965
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.276
- standardDeviation
- 0
- confidenceInterval
- 0.0554117556980965
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0554117556980965
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 46
- sort_order
- DESCENDING
- source_name
- EM Marathi
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.22058824430190352
- confidence upper
- 0.3314117556980965
Original result ↗23.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Vietnamese · EM Vietnamese
- dimension
- reasoning
- metric
- MultiLoKo Vietnamese · EM Vietnamese
- rank
- 16
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 59
- name
- EM Vietnamese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 82
- name
- MultiLoKo Vietnamese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 68
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Vietnamese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Vietnamese · EM Vietnamese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 68/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-vietnamese/versions/1
- measurements
- name
- EM Vietnamese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Vietnamese · EM Vietnamese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.236
- standardDeviation
- 0
- confidenceInterval
- 0.052635744174712
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.052635744174712
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.236
- standardDeviation
- 0
- confidenceInterval
- 0.052635744174712
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.052635744174712
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 59
- sort_order
- DESCENDING
- source_name
- EM Vietnamese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.18336425582528798
- confidence upper
- 0.288635744174712
Original result ↗36.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- dimension
- reasoning
- metric
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- rank
- 17
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 56
- name
- EM Traditional Mandarin
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 79
- name
- MultiLoKo Traditional Mandarin
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 65
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Traditional Mandarin
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 65/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-traditional-mandarin/versions/1
- measurements
- name
- EM Traditional Mandarin
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.368
- standardDeviation
- 0
- confidenceInterval
- 0.0597806365533776
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0597806365533776
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.368
- standardDeviation
- 0
- confidenceInterval
- 0.0597806365533776
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0597806365533776
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 56
- sort_order
- DESCENDING
- source_name
- EM Traditional Mandarin
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.3082193634466224
- confidence upper
- 0.4277806365533776
Original result ↗0.875
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Yoruba · Yoruba
- dimension
- reasoning
- metric
- Global MMLU Lite Yoruba · Yoruba
- rank
- 18
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41743
- name
- Yoruba
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 165
- name
- Global MMLU Lite Yoruba
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 147
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Yoruba
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Yoruba · Yoruba
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 147/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-yoruba/versions/1
- measurements
- name
- Yoruba
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Yoruba · Yoruba
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.875
- standardDeviation
- 0
- confidenceInterval
- 0.0324098580108514
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0324098580108514
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.875
- standardDeviation
- 0
- confidenceInterval
- 0.0324098580108514
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0324098580108514
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41743
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-yoruba
- sort_order
- DESCENDING
- source_name
- Yoruba
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:24.645552400Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8425901419891486
- confidence upper
- 0.9074098580108514
Original result ↗78.9%
- source id
- bfcl
- leaderboard name
- BFCL Live
- dimension
- toolUse
- metric
- Live accuracy
- rank
- 18
Original result ↗68%
- source id
- bfcl
- leaderboard name
- BFCL Web Search
- dimension
- agentic
- metric
- Web-search accuracy
- rank
- 18
Original result ↗0.878
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Hindi · Hindi
- dimension
- reasoning
- metric
- Global MMLU Lite Hindi · Hindi
- rank
- 19
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41742
- name
- Hindi
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 157
- name
- Global MMLU Lite Hindi
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 139
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Hindi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Hindi · Hindi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 139/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-hindi/versions/1
- measurements
- name
- Hindi
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Hindi · Hindi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8775
- standardDeviation
- 0
- confidenceInterval
- 0.0321299242960121
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0321299242960121
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8775
- standardDeviation
- 0
- confidenceInterval
- 0.0321299242960121
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0321299242960121
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41742
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-hindi
- sort_order
- DESCENDING
- source_name
- Hindi
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:41.662768400Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8453700757039878
- confidence upper
- 0.9096299242960121
Original result ↗0.873
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Swahili · Swahili
- dimension
- reasoning
- metric
- Global MMLU Lite Swahili · Swahili
- rank
- 20
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41750
- name
- Swahili
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 164
- name
- Global MMLU Lite Swahili
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 146
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Swahili
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Swahili · Swahili
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 146/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-swahili/versions/1
- measurements
- name
- Swahili
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Swahili · Swahili
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8725
- standardDeviation
- 0
- confidenceInterval
- 0.0326855581520567
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0326855581520567
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8725
- standardDeviation
- 0
- confidenceInterval
- 0.0326855581520567
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0326855581520567
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41750
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-swahili
- sort_order
- DESCENDING
- source_name
- Swahili
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:27.253350900Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8398144418479434
- confidence upper
- 0.9051855581520567
Original result ↗53.96%
- source id
- bfcl
- leaderboard name
- BFCL Overall
- dimension
- toolUse
- metric
- Overall accuracy
- rank
- 20
Original result ↗1,207.892
- source id
- lmarena
- leaderboard name
- LMArena Vision Captioning
- dimension
- multimodal
- metric
- Arena rating
- rank
- 21
- sample size
- 421
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- GPT-4.1 (2025-04-14)
- confidence lower
- 1176.726207800832
- confidence upper
- 1239.0582047736395
Original result ↗0.897
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Culturally Agnostic · CA Score
- dimension
- reasoning
- metric
- Global MMLU Lite Culturally Agnostic · CA Score
- rank
- 21
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41744
- name
- CA Score
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 209
- name
- Global MMLU Lite Culturally Agnostic
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 188
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Culturally Agnostic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Culturally Agnostic · CA Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 188/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-ca/versions/1
- measurements
- name
- CA Score
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Culturally Agnostic · CA Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.896875
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-03T22:31:06.2833333Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.896875
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41744
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-ca-score
- sort_order
- DESCENDING
- source_name
- CA Score
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-11-10T16:52:12.590864700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-03T22:31:06.283Z
- source model id
- GPT-4.1
Original result ↗65.993%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · OpenAI
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 21
- sample size
- 594
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- OpenAI
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-02T23:17:48.072Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 594
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.6599326666666666
- accuracy_std_dev
- 0.013255807590972711
- cost_per_task_usd
- 0.015927558922558923
- output_tokens_per_task
- 1794.7289562289561
- latency_seconds_per_task
- 14.0958367003367
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 26
- benchmark_evaluation
- true
- openrouter_permaslug
- openai/gpt-4.1-2025-04-14
- source model id
- openai/gpt-4.1-2025-04-14
Original result ↗51.333%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · OpenAI
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 21
- sample size
- 150
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- OpenAI
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-02T23:07:04.603Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 150
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.5133333333333333
- accuracy_std_dev
- 0.028803034820811636
- cost_per_task_usd
- 0.12443134666666665
- output_tokens_per_task
- 2755.9866666666667
- latency_seconds_per_task
- 52.35768
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 27
- benchmark_evaluation
- true
- openrouter_permaslug
- openai/gpt-4.1-2025-04-14
- source model id
- openai/gpt-4.1-2025-04-14
Original result ↗0.888
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite French · French
- dimension
- reasoning
- metric
- Global MMLU Lite French · French
- rank
- 22
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41752
- name
- French
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 156
- name
- Global MMLU Lite French
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 138
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite French
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite French · French
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 138/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-french/versions/1
- measurements
- name
- French
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite French · French
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8875
- standardDeviation
- 0
- confidenceInterval
- 0.0309655314070612
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0309655314070612
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8875
- standardDeviation
- 0
- confidenceInterval
- 0.0309655314070612
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0309655314070612
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41752
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-french
- sort_order
- DESCENDING
- source_name
- French
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:43.548947500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8565344685929388
- confidence upper
- 0.9184655314070611
Original result ↗0.885
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Indonesian · Indonesian
- dimension
- reasoning
- metric
- Global MMLU Lite Indonesian · Indonesian
- rank
- 22
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41740
- name
- Indonesian
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 158
- name
- Global MMLU Lite Indonesian
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 140
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Indonesian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Indonesian · Indonesian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 140/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-indonesian/versions/1
- measurements
- name
- Indonesian
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Indonesian · Indonesian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.885
- standardDeviation
- 0
- confidenceInterval
- 0.0312635759101603
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0312635759101603
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.885
- standardDeviation
- 0
- confidenceInterval
- 0.0312635759101603
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0312635759101603
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41740
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-indonesian
- sort_order
- DESCENDING
- source_name
- Indonesian
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:39.240452500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8537364240898397
- confidence upper
- 0.9162635759101603
Original result ↗0.309
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode Subproblem Standard · SciCode Subproblem Standard
- dimension
- coding
- metric
- SciCode Subproblem Standard · SciCode Subproblem Standard
- rank
- 22
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40106
- name
- SciCode Subproblem Standard
- version
- 3
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 14
- name
- SciCode Subproblem Standard
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 14
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode Subproblem Standard
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode Subproblem Standard · SciCode Subproblem Standard
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 14/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode-subproblem-standard/versions/1
- measurements
- name
- SciCode Subproblem Standard
- unit
- score
- scale
- 1
- metric
- SciCode Subproblem Standard · SciCode Subproblem Standard
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.3092783505154639
- standardDeviation
- 0
- confidenceInterval
- 0.0531040802020165
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0531040802020165
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.3092783505154639
- standardDeviation
- 0
- confidenceInterval
- 0.0531040802020165
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0531040802020165
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40106
- task_url
- /benchmarks/tasks/benchmarkler/scicode-standard-scicode-subproblem-standard
- sort_order
- DESCENDING
- source_name
- SciCode Subproblem Standard
- display_type
- rawValues
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 57
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.25617427031344736
- confidence upper
- 0.3623824307174804
Original result ↗0.885
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Spanish · Spanish
- dimension
- reasoning
- metric
- Global MMLU Lite Spanish · Spanish
- rank
- 23
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41741
- name
- Spanish
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 163
- name
- Global MMLU Lite Spanish
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 145
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Spanish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Spanish · Spanish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 145/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-spanish/versions/1
- measurements
- name
- Spanish
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Spanish · Spanish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.885
- standardDeviation
- 0
- confidenceInterval
- 0.0312635759101603
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0312635759101603
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.885
- standardDeviation
- 0
- confidenceInterval
- 0.0312635759101603
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0312635759101603
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41741
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-spanish
- sort_order
- DESCENDING
- source_name
- Spanish
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:28.873808200Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8537364240898397
- confidence upper
- 0.9162635759101603
Original result ↗0.88
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Italian · Italian
- dimension
- reasoning
- metric
- Global MMLU Lite Italian · Italian
- rank
- 23
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41739
- name
- Italian
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 159
- name
- Global MMLU Lite Italian
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 141
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Italian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Italian · Italian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 141/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-italian/versions/1
- measurements
- name
- Italian
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Italian · Italian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.88
- standardDeviation
- 0
- confidenceInterval
- 0.0318456453642134
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0318456453642134
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.88
- standardDeviation
- 0
- confidenceInterval
- 0.0318456453642134
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0318456453642134
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41739
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-italian
- sort_order
- DESCENDING
- source_name
- Italian
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:36.963753600Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8481543546357866
- confidence upper
- 0.9118456453642134
Original result ↗0.88
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Arabic · Arabic
- dimension
- reasoning
- metric
- Global MMLU Lite Arabic · Arabic
- rank
- 23
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41737
- name
- Arabic
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 148
- name
- Global MMLU Lite Arabic
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 130
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Arabic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Arabic · Arabic
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 130/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-arabic/versions/1
- measurements
- name
- Arabic
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Arabic · Arabic
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.88
- standardDeviation
- 0
- confidenceInterval
- 0.0318456453642134
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0318456453642134
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-31T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.88
- standardDeviation
- 0
- confidenceInterval
- 0.0318456453642134
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0318456453642134
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41737
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-arabic
- sort_order
- DESCENDING
- source_name
- Arabic
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:54.963186300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-31T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.8481543546357866
- confidence upper
- 0.9118456453642134
Original result ↗0.854
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Culturally Sensitive · CS Score
- dimension
- reasoning
- metric
- Global MMLU Lite Culturally Sensitive · CS Score
- rank
- 23
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41747
- name
- CS Score
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 208
- name
- Global MMLU Lite Culturally Sensitive
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 187
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Culturally Sensitive
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Culturally Sensitive · CS Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 187/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-cs/versions/1
- measurements
- name
- CS Score
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Culturally Sensitive · CS Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8540625
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-03T22:31:06.2833333Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8540625
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41747
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-cs-score
- sort_order
- DESCENDING
- source_name
- CS Score
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-11-10T16:52:15.372518Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-03T22:31:06.283Z
- source model id
- GPT-4.1
Original result ↗Published prices
1–20 / 36
$0.40
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Cached input tokens (per 1M)
$0.00
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- cache_write_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Cache write, 5 minutes (per 1M)
$1.61
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Input tokens (per 1M)
$6.46
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Output tokens (per 1M)
$0.50
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$2.00
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$8.00
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$2.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$8.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
$0.25
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$1.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$4.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
- Billing conditions
- mode
- batch
$0.50
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$2.00
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$8.00
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- request
- Currency
- USD
- Billing conditions
- charge
- web_search
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Web search
- Pricing via
- Vercel AI Gateway
$0.50
- Inference provider
- vercelAIGatewayCatalog
- Platform
- openai
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- openai | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–4 / 4
- source model id
- azure-openai/gpt-4.1@2025-04-14
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 1048576
- max output tokens
- 32768
- source model id
- gpt-4.1
- hosting
- gateway
- context length
- 1047576
- max output tokens
- 32768
- capabilities
- reasoning
- structuredOutputs
- toolCalling
- visionInput
- webSearch
- source model id
- openai/gpt-4.1
- hosting
- third-party
- context length
- 1047576
- max output tokens
- 32768
- source model id
- openai/gpt-4.1
- hosting
- vercel-gateway
- context length
- 1047576
- max output tokens
- 32768
- capabilities
- promptCaching
- structuredOutputs
- toolCalling
- visionInput
- webSearch
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/gpt-4.1-2025-04-14
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–14 / 14
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-30
- normalized value
- 2024-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-30
- normalized value
- 2024-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-01
- normalized value
- 2024-06-01
- precision
- day
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- May 2024
- normalized value
- 2024-05
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models