OpenAI: GPT-4o (2024-08-06)
The 2024-08-06 version of GPT-4o offers improved performance in structured outputs, with the ability to supply a JSON schema in the respone_format. Read more [here](https://openai.com/index/introduci…
openai/gpt-4o-2024-08-06
The 2024-08-06 version of GPT-4o offers improved performance in structured outputs, with the ability to supply a JSON schema in the respone_format. Read more [here](https://openai.com/index/introducing-structured-outputs-in-the-api/). GPT-4o ("o" for "omni") is...
Specifications & capabilities
- Input
- text, image, file
- Output
- text
- Context window · tokens
- 128000
- Knowledge cutoff
- 2023-10-31
- Capabilities
- fileInput, multimodal, promptCaching, structuredOutputs, toolCalling, visionInput
Evaluations
81–100 / 143
1.538%
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode · SciCode Main Standard
- dimension
- coding
- metric
- SciCode · SciCode Main Standard
- rank
- 41
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40105
- name
- SciCode Main Standard
- version
- 3
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 16
- name
- SciCode
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 16
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode · SciCode Main Standard
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 16/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode/versions/1
- measurements
- name
- SciCode Main Standard
- unit
- percent
- scale
- 1
- metric
- SciCode · SciCode Main Standard
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0153846153846153
- standardDeviation
- 0
- confidenceInterval
- 0.0299204446309948
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0299204446309948
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0153846153846153
- standardDeviation
- 0
- confidenceInterval
- 0.0299204446309948
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0299204446309948
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40105
- task_url
- /benchmarks/open-benchmarks/scicode-main-standard/versions/1
- sort_order
- DESCENDING
- source_name
- SciCode Main Standard
- display_type
- percentages
- task_version
- 3
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-10-21T20:29:26.251536900Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 62
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4o
- confidence lower
- -0.014535829246379499
- confidence upper
- 0.0453050600156101
Original result ↗72.989%
- source id
- kaggleBenchmarks
- leaderboard name
- MMLU-Pro · mmlu_pro_task
- dimension
- reasoning
- metric
- MMLU-Pro · mmlu_pro_task
- rank
- 44
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40138
- name
- mmlu_pro_task
- version
- 12
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 44
- name
- MMLU-Pro
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- version_id
- 30
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MMLU-Pro
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MMLU-Pro · mmlu_pro_task
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 30/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mmlu-pro/versions/1
- measurements
- name
- mmlu_pro_task
- unit
- percent
- scale
- 1
- metric
- MMLU-Pro · mmlu_pro_task
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.7298869680851063
- standardDeviation
- 0
- confidenceInterval
- 0.0079337726835822
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0079337726835822
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-13T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.7298869680851063
- standardDeviation
- 0
- confidenceInterval
- 0.0079337726835822
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0079337726835822
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40138
- task_url
- /benchmarks/tasks/benchmarkler/mmlu-pro-task
- sort_order
- DESCENDING
- source_name
- mmlu_pro_task
- display_type
- percentages
- task_version
- 12
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 64
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-13T00:00:00.000Z
- source model id
- GPT-4o
- confidence lower
- 0.7219531954015241
- confidence upper
- 0.7378207407686885
Original result ↗33.677%
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode · SciCode Subproblem With Background
- dimension
- coding
- metric
- SciCode · SciCode Subproblem With Background
- rank
- 44
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40200
- name
- SciCode Subproblem With Background
- version
- 3
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 16
- name
- SciCode
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 16
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode · SciCode Subproblem With Background
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 16/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode/versions/1
- measurements
- name
- SciCode Subproblem With Background
- unit
- percent
- scale
- 1
- metric
- SciCode · SciCode Subproblem With Background
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.3367697594501718
- standardDeviation
- 0
- confidenceInterval
- 0.0543000588021355
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0543000588021355
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.3367697594501718
- standardDeviation
- 0
- confidenceInterval
- 0.0543000588021355
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0543000588021355
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40200
- task_url
- /benchmarks/open-benchmarks/scicode-subproblem-with-background/versions/1
- sort_order
- DESCENDING
- source_name
- SciCode Subproblem With Background
- display_type
- percentages
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-21T20:29:26.251536900Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 62
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4o
- confidence lower
- 0.2824697006480363
- confidence upper
- 0.3910698182523073
Original result ↗24.055%
- source id
- kaggleBenchmarks
- leaderboard name
- SciCode · SciCode Subproblem Standard
- dimension
- coding
- metric
- SciCode · SciCode Subproblem Standard
- rank
- 44
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40106
- name
- SciCode Subproblem Standard
- version
- 3
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 16
- name
- SciCode
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- coding
- science
- version_id
- 16
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · SciCode
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SciCode · SciCode Subproblem Standard
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 16/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/scicode/versions/1
- measurements
- name
- SciCode Subproblem Standard
- unit
- percent
- scale
- 1
- metric
- SciCode · SciCode Subproblem Standard
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.2405498281786941
- standardDeviation
- 0
- confidenceInterval
- 0.0491081715605116
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0491081715605116
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-22T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.2405498281786941
- standardDeviation
- 0
- confidenceInterval
- 0.0491081715605116
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0491081715605116
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40106
- task_url
- /benchmarks/open-benchmarks/scicode-subproblem-standard/versions/1
- sort_order
- DESCENDING
- source_name
- SciCode Subproblem Standard
- display_type
- percentages
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-21T20:29:26.251536900Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 62
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-22T00:00:00.000Z
- source model id
- GPT-4o
- confidence lower
- 0.19144165661818252
- confidence upper
- 0.2896579997392057
Original result ↗5.833%
- source id
- kaggleBenchmarks
- leaderboard name
- AIME 2025 · aime_task
- dimension
- math
- metric
- AIME 2025 · aime_task
- rank
- 58
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40087
- name
- aime_task
- version
- 6
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 125
- name
- AIME 2025
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- math
- version_id
- 111
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · AIME 2025
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AIME 2025 · aime_task
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 111/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/aime-2025/versions/1
- measurements
- name
- aime_task
- unit
- percent
- scale
- 1
- metric
- AIME 2025 · aime_task
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0583333333333333
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-07-07T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0583333333333333
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40087
- task_url
- /benchmarks/tasks/benchmarkler/aime-task
- sort_order
- DESCENDING
- source_name
- aime_task
- display_type
- percentages
- task_version
- 6
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 68
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-07-07T00:00:00.000Z
- source model id
- GPT-4o
Original result ↗21.67%
- source id
- sweBench
- leaderboard name
- SWE-bench Lite
- dimension
- coding
- metric
- Resolved tasks
- rank
- 72
- evaluated at
- 2024-08-28T00:00:00.000Z
Original result ↗49.2%
- source id
- sweBench
- leaderboard name
- SWE-bench Verified
- dimension
- coding
- metric
- Resolved tasks
- rank
- 111
- evaluated at
- 2024-11-05T00:00:00.000Z
Original result ↗42.667%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · auto-routing
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 116
- sample size
- 100
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- auto-routing
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-01T16:00:49.880Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 100
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.42666650000000006
- accuracy_std_dev
- 0.00666650000000002
- cost_per_task_usd
- 0.21223042999999997
- output_tokens_per_task
- 2189.48
- latency_seconds_per_task
- 51.6851
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 128
- benchmark_evaluation
- true
- openrouter_permaslug
- openai/gpt-4o-2024-08-06
- source model id
- openai/gpt-4o-2024-08-06
Original result ↗46.6%
- source id
- sweBench
- leaderboard name
- SWE-bench Verified
- dimension
- coding
- metric
- Resolved tasks
- rank
- 117
- evaluated at
- 2024-10-23T00:00:00.000Z
Original result ↗52.02%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · auto-routing
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 121
- sample size
- 396
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- auto-routing
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-01T16:04:12.703Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 396
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.5202019999999999
- accuracy_std_dev
- 0.0033669999999999534
- cost_per_task_usd
- 0.017511833333333334
- output_tokens_per_task
- 1559.209595959596
- latency_seconds_per_task
- 16.17711868686869
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 132
- benchmark_evaluation
- true
- openrouter_permaslug
- openai/gpt-4o-2024-08-06
- source model id
- openai/gpt-4o-2024-08-06
Original result ↗41.6%
- source id
- sweBench
- leaderboard name
- SWE-bench Verified
- dimension
- coding
- metric
- Resolved tasks
- rank
- 131
- evaluated at
- 2024-10-30T00:00:00.000Z
Original result ↗7.7
- source id
- artificialAnalysis
- dimension
- intelligence
- metric
- Intelligence Index
Original result ↗29.853%
- source id
- kaggleBenchmarks
- leaderboard name
- ECLeKTic · Score
- dimension
- knowledge
- metric
- ECLeKTic · Score
- rank
- 7
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 20
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 41
- name
- ECLeKTic
- type
- INDIVIDUAL
- owner
- google
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- factuality and grounding
- version_id
- 27
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · google · ECLeKTic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- ECLeKTic · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 27/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/google/eclektic/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- ECLeKTic · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.2985321969696969
- standardDeviation
- 0
- confidenceInterval
- 0.0276004136815052
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0276004136815052
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-14T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.2985321969696969
- standardDeviation
- 0
- confidenceInterval
- 0.0276004136815052
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0276004136815052
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 20
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 35
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-14T00:00:00.000Z
- source model id
- GPT-4o
- confidence lower
- 0.27093178328819173
- confidence upper
- 0.3261326106512021
Original result ↗60.654%
- source id
- kaggleBenchmarks
- leaderboard name
- ECLeKTic · Transfer Score
- dimension
- knowledge
- metric
- ECLeKTic · Transfer Score
- rank
- 9
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 256
- name
- Transfer Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 41
- name
- ECLeKTic
- type
- INDIVIDUAL
- owner
- google
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- factuality and grounding
- version_id
- 27
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · google · ECLeKTic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- ECLeKTic · Transfer Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 27/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/google/eclektic/versions/1
- measurements
- name
- Transfer Score
- unit
- percent
- scale
- 1
- metric
- ECLeKTic · Transfer Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.6065416065416065
- standardDeviation
- 0
- confidenceInterval
- 0.0419981874700787
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0419981874700787
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-05-14T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.6065416065416065
- standardDeviation
- 0
- confidenceInterval
- 0.0419981874700787
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0419981874700787
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 256
- sort_order
- DESCENDING
- source_name
- Transfer Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 34
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-05-14T00:00:00.000Z
- source model id
- GPT-4o
- confidence lower
- 0.5645434190715278
- confidence upper
- 0.6485397940116852
Original result ↗88.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM Bengali · Score
- dimension
- math
- metric
- MGSM Bengali · Score
- rank
- 13
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 121
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 114
- name
- MGSM Bengali
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 100
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM Bengali
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM Bengali · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 100/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm-bengali/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- MGSM Bengali · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.888
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.888
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 121
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- GPT-4o
Original result ↗88.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM: Multilingual Grade School Math Benchmark · MGSM Bengali
- dimension
- math
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM Bengali
- rank
- 13
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 121
- name
- MGSM Bengali
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 113
- name
- MGSM: Multilingual Grade School Math Benchmark
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 99
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM: Multilingual Grade School Math Benchmark
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM: Multilingual Grade School Math Benchmark · MGSM Bengali
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 99/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm/versions/1
- measurements
- name
- MGSM Bengali
- unit
- percent
- scale
- 1
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM Bengali
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.888
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.888
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 121
- task_url
- /benchmarks/open-benchmarks/mgsm-bengali/versions/1
- sort_order
- DESCENDING
- source_name
- MGSM Bengali
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- GPT-4o
Original result ↗93.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM: Multilingual Grade School Math Benchmark · MGSM Thai
- dimension
- math
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM Thai
- rank
- 18
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 130
- name
- MGSM Thai
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 113
- name
- MGSM: Multilingual Grade School Math Benchmark
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 99
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM: Multilingual Grade School Math Benchmark
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM: Multilingual Grade School Math Benchmark · MGSM Thai
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 99/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm/versions/1
- measurements
- name
- MGSM Thai
- unit
- percent
- scale
- 1
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM Thai
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.936
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.936
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 130
- task_url
- /benchmarks/open-benchmarks/mgsm-thai/versions/1
- sort_order
- DESCENDING
- source_name
- MGSM Thai
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- GPT-4o
Original result ↗93.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM Thai · Score
- dimension
- math
- metric
- MGSM Thai · Score
- rank
- 18
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 130
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 123
- name
- MGSM Thai
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 109
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM Thai
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM Thai · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 109/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm-thai/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- MGSM Thai · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.936
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.936
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 130
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- GPT-4o
Original result ↗97.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM English · Score
- dimension
- math
- metric
- MGSM English · Score
- rank
- 19
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 123
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 116
- name
- MGSM English
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 102
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM English
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM English · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 102/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm-english/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- MGSM English · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.972
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.972
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 123
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- GPT-4o
Original result ↗97.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM: Multilingual Grade School Math Benchmark · MGSM English
- dimension
- math
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM English
- rank
- 19
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 123
- name
- MGSM English
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 113
- name
- MGSM: Multilingual Grade School Math Benchmark
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 99
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM: Multilingual Grade School Math Benchmark
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM: Multilingual Grade School Math Benchmark · MGSM English
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 99/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm/versions/1
- measurements
- name
- MGSM English
- unit
- percent
- scale
- 1
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM English
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.972
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.972
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 123
- task_url
- /benchmarks/open-benchmarks/mgsm-english/versions/1
- sort_order
- DESCENDING
- source_name
- MGSM English
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4o-2024-08-06
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- GPT-4o
Original result ↗Published prices
1–3 / 3
$1.25
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$2.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$10.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/gpt-4o-2024-08-06
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–3 / 3
- field
- context_length
- value
- 128000
- normalized value
- 128000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2023-10-31
- normalized value
- 2023-10-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 16384
- normalized value
- 16384
- scope
- source-model
- source id
- openRouter
- confidence
- 1
Explore more models