OpenAI: GPT-4.1
GPT-4.1 is a flagship large language model optimized for advanced instruction following, real-world software engineering, and long-context reasoning. It supports a 1 million token context window and …
openai/gpt-4.1-2025-04-14
GPT-4.1 is a flagship large language model optimized for advanced instruction following, real-world software engineering, and long-context reasoning. It supports a 1 million token context window and outperforms GPT-4o and GPT-4.5 across coding (54.6% SWE-bench Verified), instruction compliance (87.4% IFEval), and multimodal understanding benchmarks. It is tuned for precise code diffs, agent reliability, and high recall in large document contexts, making it ideal for agents, IDE tooling, and enterprise knowledge retrieval.
Specifications & capabilities
- Input
- image, text, file
- Output
- text
- Context window · tokens
- 1047576
- Published model ID
- gpt-4.1-2025-04-14
- Capabilities
- fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗Evaluations
1–20 / 257
46%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · FakeProvider
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 1
- sample size
- 100
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- FakeProvider
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-01T19:06:51.244Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 100
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.46
- accuracy_std_dev
- 0
- output_tokens_per_task
- 4515
- latency_seconds_per_task
- 158.12484
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 1
- benchmark_evaluation
- true
- openrouter_permaslug
- openai/gpt-4.1-2025-04-14
- source model id
- openai/gpt-4.1-2025-04-14
Original result ↗100%
- source id
- bfcl
- leaderboard name
- BFCL Relevance Detection
- dimension
- toolUse
- metric
- Relevance detection
- rank
- 3
Original result ↗59.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo English · EM English
- dimension
- reasoning
- metric
- MultiLoKo English · EM English
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 27
- name
- EM English
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 52
- name
- MultiLoKo English
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 38
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo English
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo English · EM English
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 38/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-english/versions/1
- measurements
- name
- EM English
- unit
- percent
- scale
- 1
- metric
- MultiLoKo English · EM English
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.596
- standardDeviation
- 0
- confidenceInterval
- 0.0608263699617859
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0608263699617859
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.596
- standardDeviation
- 0
- confidenceInterval
- 0.0608263699617859
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0608263699617859
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 27
- sort_order
- DESCENDING
- source_name
- EM English
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.535173630038214
- confidence upper
- 0.6568263699617859
Original result ↗32.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Bengali · EM Bengali
- dimension
- reasoning
- metric
- MultiLoKo Bengali · EM Bengali
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 31
- name
- EM Bengali
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 54
- name
- MultiLoKo Bengali
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 40
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Bengali
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Bengali · EM Bengali
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 40/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-bengali/versions/1
- measurements
- name
- EM Bengali
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Bengali · EM Bengali
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.324
- standardDeviation
- 0
- confidenceInterval
- 0.0580128150237069
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0580128150237069
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.324
- standardDeviation
- 0
- confidenceInterval
- 0.0580128150237069
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0580128150237069
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 31
- sort_order
- DESCENDING
- source_name
- EM Bengali
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.2659871849762931
- confidence upper
- 0.38201281502370693
Original result ↗3.555%
- source id
- kaggleBenchmarks
- leaderboard name
- BrowseComp · Score
- dimension
- knowledge
- metric
- BrowseComp · Score
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 26
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 49
- name
- BrowseComp
- type
- INDIVIDUAL
- owner
- openai
- version
- 1
- categories
- general knowledge and reasoning
- factuality and grounding
- version_id
- 35
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · openai · BrowseComp
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- BrowseComp · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 35/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/openai/browsecomp/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- BrowseComp · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0355450236966824
- standardDeviation
- 0
- confidenceInterval
- 0.0101990946312181
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0101990946312181
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-13T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0355450236966824
- standardDeviation
- 0
- confidenceInterval
- 0.0101990946312181
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0101990946312181
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 26
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T14:25:30.827974100Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 42
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-13T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.025345929065464304
- confidence upper
- 0.0457441183279005
Original result ↗44.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Tagalog · EM Tagalog
- dimension
- reasoning
- metric
- MultiLoKo Tagalog · EM Tagalog
- rank
- 7
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 54
- name
- EM Tagalog
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 77
- name
- MultiLoKo Tagalog
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 63
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Tagalog
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Tagalog · EM Tagalog
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 63/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-tagalog/versions/1
- measurements
- name
- EM Tagalog
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Tagalog · EM Tagalog
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.448
- standardDeviation
- 0
- confidenceInterval
- 0.0616434068014536
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0616434068014536
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.448
- standardDeviation
- 0
- confidenceInterval
- 0.0616434068014536
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0616434068014536
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 54
- sort_order
- DESCENDING
- source_name
- EM Tagalog
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.3863565931985464
- confidence upper
- 0.5096434068014536
Original result ↗36%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Swedish · EM Swedish
- dimension
- reasoning
- metric
- MultiLoKo Swedish · EM Swedish
- rank
- 7
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 53
- name
- EM Swedish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 76
- name
- MultiLoKo Swedish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 62
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Swedish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Swedish · EM Swedish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 62/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-swedish/versions/1
- measurements
- name
- EM Swedish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Swedish · EM Swedish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.36
- standardDeviation
- 0
- confidenceInterval
- 0.0595003231012379
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0595003231012379
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.36
- standardDeviation
- 0
- confidenceInterval
- 0.0595003231012379
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0595003231012379
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 53
- sort_order
- DESCENDING
- source_name
- EM Swedish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.3004996768987621
- confidence upper
- 0.41950032310123786
Original result ↗31.14%
- source id
- sweBench
- leaderboard name
- SWE-bench Multimodal
- dimension
- coding
- metric
- Resolved tasks
- rank
- 7
- evaluated at
- 2025-05-31T00:00:00.000Z
Original result ↗42.47
- source id
- kaggleBenchmarks
- leaderboard name
- IndicGenBench · Flores-In
- dimension
- general
- metric
- IndicGenBench · Flores-In
- rank
- 8
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 816
- name
- Flores-In
- version
- 1
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 212
- name
- IndicGenBench
- type
- SUITE
- owner
- deepmind
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- version_id
- 191
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · deepmind · IndicGenBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- IndicGenBench · Flores-In
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 191/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/deepmind/indic-gen-bench/versions/1
- measurements
- name
- Flores-In
- unit
- score
- scale
- 1
- metric
- IndicGenBench · Flores-In
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 42.46999598416758
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-28T22:12:09.220259Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 42.46999598416758
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 816
- task_url
- /benchmarks/tasks/aminmohamedmohami/flores-in-task
- sort_order
- DESCENDING
- source_name
- Flores-In
- display_type
- counts
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-04T15:09:28.942941700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 13
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-28T22:12:09.220Z
- source model id
- GPT-4.1
Original result ↗18.263
- source id
- kaggleBenchmarks
- leaderboard name
- IndicGenBench · CrossSum-In
- dimension
- general
- metric
- IndicGenBench · CrossSum-In
- rank
- 8
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 815
- name
- CrossSum-In
- version
- 1
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 212
- name
- IndicGenBench
- type
- SUITE
- owner
- deepmind
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- version_id
- 191
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · deepmind · IndicGenBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- IndicGenBench · CrossSum-In
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 191/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/deepmind/indic-gen-bench/versions/1
- measurements
- name
- CrossSum-In
- unit
- score
- scale
- 1
- metric
- IndicGenBench · CrossSum-In
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 18.262619854115293
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-28T22:12:58.270099Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 18.262619854115293
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 815
- task_url
- /benchmarks/tasks/aminmohamedmohami/crosssum-in-task
- sort_order
- DESCENDING
- source_name
- CrossSum-In
- display_type
- counts
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-04T15:09:28.942941700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 13
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-28T22:12:58.270Z
- source model id
- GPT-4.1
Original result ↗44.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Spanish · EM Spanish
- dimension
- reasoning
- metric
- MultiLoKo Spanish · EM Spanish
- rank
- 8
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 52
- name
- EM Spanish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 75
- name
- MultiLoKo Spanish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 61
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Spanish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Spanish · EM Spanish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 61/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-spanish/versions/1
- measurements
- name
- EM Spanish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Spanish · EM Spanish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.444
- standardDeviation
- 0
- confidenceInterval
- 0.0615895410053309
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0615895410053309
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.444
- standardDeviation
- 0
- confidenceInterval
- 0.0615895410053309
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0615895410053309
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 52
- sort_order
- DESCENDING
- source_name
- EM Spanish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.3824104589946691
- confidence upper
- 0.5055895410053309
Original result ↗29.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Arabic · EM Arabic
- dimension
- reasoning
- metric
- MultiLoKo Arabic · EM Arabic
- rank
- 8
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 30
- name
- EM Arabic
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 53
- name
- MultiLoKo Arabic
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 39
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Arabic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Arabic · EM Arabic
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 39/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-arabic/versions/1
- measurements
- name
- EM Arabic
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Arabic · EM Arabic
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.292
- standardDeviation
- 0
- confidenceInterval
- 0.0563619669903391
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0563619669903391
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.292
- standardDeviation
- 0
- confidenceInterval
- 0.0563619669903391
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0563619669903391
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 30
- sort_order
- DESCENDING
- source_name
- EM Arabic
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.23563803300966088
- confidence upper
- 0.34836196699033906
Original result ↗61.036
- source id
- kaggleBenchmarks
- leaderboard name
- IndicGenBench · XorQA-In
- dimension
- general
- metric
- IndicGenBench · XorQA-In
- rank
- 9
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 817
- name
- XorQA-In
- version
- 1
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 212
- name
- IndicGenBench
- type
- SUITE
- owner
- deepmind
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- version_id
- 191
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · deepmind · IndicGenBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- IndicGenBench · XorQA-In
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 191/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/deepmind/indic-gen-bench/versions/1
- measurements
- name
- XorQA-In
- unit
- score
- scale
- 1
- metric
- IndicGenBench · XorQA-In
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 61.03601568977308
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-28T22:12:23.731366Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 61.03601568977308
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 817
- task_url
- /benchmarks/tasks/aminmohamedmohami/xorqa-in-task
- sort_order
- DESCENDING
- source_name
- XorQA-In
- display_type
- counts
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-04T15:09:28.942941700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 13
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-28T22:12:23.731Z
- source model id
- GPT-4.1
Original result ↗43.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Czech · EM Czech
- dimension
- reasoning
- metric
- MultiLoKo Czech · EM Czech
- rank
- 9
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 33
- name
- EM Czech
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 56
- name
- MultiLoKo Czech
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 42
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Czech
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Czech · EM Czech
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 42/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-czech/versions/1
- measurements
- name
- EM Czech
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Czech · EM Czech
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.436
- standardDeviation
- 0
- confidenceInterval
- 0.061469670239687
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.061469670239687
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.436
- standardDeviation
- 0
- confidenceInterval
- 0.061469670239687
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.061469670239687
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 33
- sort_order
- DESCENDING
- source_name
- EM Czech
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.374530329760313
- confidence upper
- 0.497469670239687
Original result ↗37.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Dutch · EM Dutch
- dimension
- reasoning
- metric
- MultiLoKo Dutch · EM Dutch
- rank
- 9
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 34
- name
- EM Dutch
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 57
- name
- MultiLoKo Dutch
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 43
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Dutch
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Dutch · EM Dutch
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 43/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-dutch/versions/1
- measurements
- name
- EM Dutch
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Dutch · EM Dutch
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.372
- standardDeviation
- 0
- confidenceInterval
- 0.0599141467052741
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0599141467052741
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.372
- standardDeviation
- 0
- confidenceInterval
- 0.0599141467052741
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0599141467052741
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 34
- sort_order
- DESCENDING
- source_name
- EM Dutch
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.3120858532947259
- confidence upper
- 0.4319141467052741
Original result ↗34.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Japanese · EM Japanese
- dimension
- reasoning
- metric
- MultiLoKo Japanese · EM Japanese
- rank
- 9
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 42
- name
- EM Japanese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 65
- name
- MultiLoKo Japanese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 51
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Japanese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Japanese · EM Japanese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 51/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-japanese/versions/1
- measurements
- name
- EM Japanese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Japanese · EM Japanese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.344
- standardDeviation
- 0
- confidenceInterval
- 0.0588856166924697
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0588856166924697
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.344
- standardDeviation
- 0
- confidenceInterval
- 0.0588856166924697
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0588856166924697
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 42
- sort_order
- DESCENDING
- source_name
- EM Japanese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.28511438330753025
- confidence upper
- 0.4028856166924697
Original result ↗32.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Turkish · EM Turkish
- dimension
- reasoning
- metric
- MultiLoKo Turkish · EM Turkish
- rank
- 9
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 57
- name
- EM Turkish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 80
- name
- MultiLoKo Turkish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 66
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Turkish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Turkish · EM Turkish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 66/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-turkish/versions/1
- measurements
- name
- EM Turkish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Turkish · EM Turkish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.324
- standardDeviation
- 0
- confidenceInterval
- 0.0580128150237069
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0580128150237069
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.324
- standardDeviation
- 0
- confidenceInterval
- 0.0580128150237069
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0580128150237069
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 57
- sort_order
- DESCENDING
- source_name
- EM Turkish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.2659871849762931
- confidence upper
- 0.38201281502370693
Original result ↗29.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Hebrew · EM Hebrew
- dimension
- reasoning
- metric
- MultiLoKo Hebrew · EM Hebrew
- rank
- 9
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 38
- name
- EM Hebrew
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 61
- name
- MultiLoKo Hebrew
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 47
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Hebrew
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Hebrew · EM Hebrew
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 47/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-hebrew/versions/1
- measurements
- name
- EM Hebrew
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Hebrew · EM Hebrew
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.292
- standardDeviation
- 0
- confidenceInterval
- 0.0563619669903391
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0563619669903391
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.292
- standardDeviation
- 0
- confidenceInterval
- 0.0563619669903391
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0563619669903391
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 38
- sort_order
- DESCENDING
- source_name
- EM Hebrew
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- GPT-4.1
- confidence lower
- 0.23563803300966088
- confidence upper
- 0.34836196699033906
Original result ↗52.859
- source id
- kaggleBenchmarks
- leaderboard name
- IndicGenBench · XQuAD-In
- dimension
- general
- metric
- IndicGenBench · XQuAD-In
- rank
- 10
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 820
- name
- XQuAD-In
- version
- 3
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 212
- name
- IndicGenBench
- type
- SUITE
- owner
- deepmind
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- version_id
- 191
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · deepmind · IndicGenBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- IndicGenBench · XQuAD-In
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 191/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/deepmind/indic-gen-bench/versions/1
- measurements
- name
- XQuAD-In
- unit
- score
- scale
- 1
- metric
- IndicGenBench · XQuAD-In
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 52.8586213265185
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-29T00:38:02.279539Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 52.8586213265185
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 820
- task_url
- /benchmarks/tasks/aminmohamedmohami/xquad-in-task
- sort_order
- DESCENDING
- source_name
- XQuAD-In
- display_type
- counts
- task_version
- 3
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-04T15:09:28.942941700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 14
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-29T00:38:02.279Z
- source model id
- GPT-4.1
Original result ↗43.657
- source id
- kaggleBenchmarks
- leaderboard name
- IndicGenBench · Average
- dimension
- general
- metric
- IndicGenBench · Average
- rank
- 10
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 897
- name
- Average
- version
- 1
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 212
- name
- IndicGenBench
- type
- SUITE
- owner
- deepmind
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- version_id
- 191
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · deepmind · IndicGenBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- IndicGenBench · Average
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 191/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/deepmind/indic-gen-bench/versions/1
- measurements
- name
- Average
- unit
- score
- scale
- 1
- metric
- IndicGenBench · Average
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 43.656813213643616
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-29T00:38:02.279539Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 43.656813213643616
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 897
- sort_order
- DESCENDING
- source_name
- Average
- display_type
- counts
- task_version
- 1
- task_display_type
- COUNTS
- normalization_status
- normalized
- published_at
- 2025-12-04T15:09:28.942941700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/gpt-4.1-2025-04-14
- metric_population
- 14
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-29T00:38:02.279Z
- source model id
- GPT-4.1
Original result ↗Published prices
1–20 / 36
$0.40
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Cached input tokens (per 1M)
$0.00
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- cache_write_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Cache write, 5 minutes (per 1M)
$1.61
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Input tokens (per 1M)
$6.46
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Output tokens (per 1M)
$0.50
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$2.00
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$8.00
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$2.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$8.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
$0.25
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$1.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$4.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
- Billing conditions
- mode
- batch
$0.50
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$2.00
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$8.00
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- request
- Currency
- USD
- Billing conditions
- charge
- web_search
- endpoint
- azure | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Web search
- Pricing via
- Vercel AI Gateway
$0.50
- Inference provider
- vercelAIGatewayCatalog
- Platform
- openai
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- openai | openai/gpt-4.1
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–4 / 4
- source model id
- azure-openai/gpt-4.1@2025-04-14
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 1048576
- max output tokens
- 32768
- source model id
- gpt-4.1
- hosting
- gateway
- context length
- 1047576
- max output tokens
- 32768
- capabilities
- reasoning
- structuredOutputs
- toolCalling
- visionInput
- webSearch
- source model id
- openai/gpt-4.1
- hosting
- third-party
- context length
- 1047576
- max output tokens
- 32768
- source model id
- openai/gpt-4.1
- hosting
- vercel-gateway
- context length
- 1047576
- max output tokens
- 32768
- capabilities
- promptCaching
- structuredOutputs
- toolCalling
- visionInput
- webSearch
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/gpt-4.1-2025-04-14
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–14 / 14
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1047576
- normalized value
- 1047576
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-30
- normalized value
- 2024-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-30
- normalized value
- 2024-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-01
- normalized value
- 2024-06-01
- precision
- day
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- May 2024
- normalized value
- 2024-05
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models