Meta: Llama 4 Maverick
Llama 4 Maverick 17B Instruct (128E) is a high-capacity multimodal language model from Meta, built on a mixture-of-experts (MoE) architecture with 128 experts and 17 billion active parameters per for…
meta-llama/llama-4-maverick-17b-128e-instruct
Llama 4 Maverick 17B Instruct (128E) is a high-capacity multimodal language model from Meta, built on a mixture-of-experts (MoE) architecture with 128 experts and 17 billion active parameters per forward pass (400B total). It supports multilingual text and image input, and produces multilingual text and code output across 12 supported languages. Optimized for vision-language tasks, Maverick is instruction-tuned for assistant-like behavior, image reasoning, and general-purpose multimodal interaction.
Maverick features early fusion for native multimodality and a 1 million token context window. It was trained on a curated mixture of public, licensed, and Meta-platform data, covering ~22 trillion tokens, with a knowledge cutoff in August 2024. Released on April 5, 2025 under the Llama 4 Community License, Maverick is suited for research and commercial applications requiring advanced multimodal understanding and high model throughput.
Specifications & capabilities
- Input
- text, image
- Output
- text
- Context window · tokens
- 524288
- Knowledge cutoff
- 2024-08-31
- Published model ID
- Llama-4-Maverick-17B-128E-Instruct
- Capabilities
- multimodal, promptCaching, structuredOutputs, toolCalling, visionInput
Description source ↗Evaluations
1–20 / 102
0.776
- source id
- llmStatsArchive
- leaderboard name
- MBPP · Normalized (reported)
- dimension
- general
- metric
- MBPP · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MBPP
- language
- en
- modality
- text
- verified
- false
- max_score
- 100
- created_at
- 2025-07-19T19:56:13.453370+00:00
- paper_link
- https://arxiv.org/abs/2108.07732
- updated_at
- 2025-07-19T19:56:13.453370+00:00
- description
- MBPP (Mostly Basic Python Problems) is a benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers. Each problem consists of a task description, code solution, and 3 automated test cases covering programming fundamentals and standard library functionality.
- benchmark_id
- mbpp
- multilingual
- false
- analysis_method
- 3-shot pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MBPP · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/meta/models/llama-4-maverick/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- meta/llama-4-maverick
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- llama-4-maverick
Original result ↗0.776
- source id
- llmStatsArchive
- leaderboard name
- MBPP · Raw
- dimension
- general
- metric
- MBPP · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MBPP
- language
- en
- modality
- text
- verified
- false
- max_score
- 100
- created_at
- 2025-07-19T19:56:13.453370+00:00
- paper_link
- https://arxiv.org/abs/2108.07732
- updated_at
- 2025-07-19T19:56:13.453370+00:00
- description
- MBPP (Mostly Basic Python Problems) is a benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers. Each problem consists of a task description, code solution, and 3 automated test cases covering programming fundamentals and standard library functionality.
- benchmark_id
- mbpp
- multilingual
- false
- analysis_method
- 3-shot pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MBPP · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/meta/models/llama-4-maverick/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- meta/llama-4-maverick
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- llama-4-maverick
Original result ↗0.317
- source id
- llmStatsArchive
- leaderboard name
- TydiQA · Normalized (reported)
- dimension
- general
- metric
- TydiQA · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- TydiQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:14.470500+00:00
- paper_link
- https://arxiv.org/abs/2003.05002
- updated_at
- 2025-07-19T19:56:14.470500+00:00
- description
- A multilingual question answering benchmark covering 11 typologically diverse languages with 204K question-answer pairs. Questions are written by people seeking genuine information and data is collected directly in each language without translation to test model generalization across diverse linguistic structures.
- benchmark_id
- tydiqa
- multilingual
- true
- analysis_method
- 1-shot average/f1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- TydiQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/meta/models/llama-4-maverick/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- meta/llama-4-maverick
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- llama-4-maverick
Original result ↗0.317
- source id
- llmStatsArchive
- leaderboard name
- TydiQA · Raw
- dimension
- general
- metric
- TydiQA · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- TydiQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:14.470500+00:00
- paper_link
- https://arxiv.org/abs/2003.05002
- updated_at
- 2025-07-19T19:56:14.470500+00:00
- description
- A multilingual question answering benchmark covering 11 typologically diverse languages with 204K question-answer pairs. Questions are written by people seeking genuine information and data is collected directly in each language without translation to test model generalization across diverse linguistic structures.
- benchmark_id
- tydiqa
- multilingual
- true
- analysis_method
- 1-shot average/f1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- TydiQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/meta/models/llama-4-maverick/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- meta/llama-4-maverick
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- llama-4-maverick
Original result ↗41.441%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · DigitalOcean
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 20
- sample size
- 148
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- DigitalOcean
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-05T20:09:54.588Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 148
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.4144140810810811
- accuracy_std_dev
- 0.03472111109333275
- cost_per_task_usd
- 0.07118006081081081
- output_tokens_per_task
- 2651.4324324324325
- latency_seconds_per_task
- 303.4494054054054
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 20
- benchmark_evaluation
- true
- openrouter_permaslug
- meta-llama/llama-4-maverick-17b-128e-instruct
- source model id
- meta-llama/llama-4-maverick-17b-128e-instruct
Original result ↗66.386%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · DigitalOcean
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 21
- sample size
- 594
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- DigitalOcean
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-05T20:44:48.611Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 594
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.663861
- accuracy_std_dev
- 0.007815979699734826
- cost_per_task_usd
- 0.002078132996632997
- output_tokens_per_task
- 3140.030303030303
- latency_seconds_per_task
- 241.27114814814814
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 23
- benchmark_evaluation
- true
- openrouter_permaslug
- meta-llama/llama-4-maverick-17b-128e-instruct
- source model id
- meta-llama/llama-4-maverick-17b-128e-instruct
Original result ↗41.837%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Parasail
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 27
- sample size
- 98
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Parasail
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-08-29T16:52:55.844Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 98
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.4183672857142857
- accuracy_std_dev
- 0.0459725
- cost_per_task_usd
- 0.06366735714285714
- output_tokens_per_task
- 2676.5102040816328
- latency_seconds_per_task
- 88.35082653061225
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 28
- benchmark_evaluation
- true
- openrouter_permaslug
- meta-llama/llama-4-maverick-17b-128e-instruct
- source model id
- meta-llama/llama-4-maverick-17b-128e-instruct
Original result ↗79.583%
- source id
- liveBench
- leaderboard name
- LiveBench Instruction Following
- dimension
- general
- metric
- Average score
- rank
- 31
- sample size
- 50
- evaluated at
- 2025-04-06T00:20:17.613Z
- source model id
- llama-4-maverick-17b-128e-instruct
Original result ↗79.6%
- source id
- openCompass
- leaderboard name
- MMLU-Pro
- dimension
- general
- metric
- MMLU-Pro
- rank
- 32
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- MMLU-Pro
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Llama4-Maverick-17B-128E-Instruct
Original result ↗65.236%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Parasail
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 32
- sample size
- 396
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Parasail
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-08-29T19:51:51.198Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 396
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.6523570000000002
- accuracy_std_dev
- 0.01094299999999998
- cost_per_task_usd
- 0.003023964646464647
- output_tokens_per_task
- 2857.7727272727275
- latency_seconds_per_task
- 45.756330808080804
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 36
- benchmark_evaluation
- true
- openrouter_permaslug
- meta-llama/llama-4-maverick-17b-128e-instruct
- source model id
- meta-llama/llama-4-maverick-17b-128e-instruct
Original result ↗52.398%
- source id
- liveBench
- leaderboard name
- LiveBench Overall
- dimension
- general
- metric
- Average score
- rank
- 32
- sample size
- 380
- evaluated at
- 2025-04-07T16:07:43.128Z
- source model id
- llama-4-maverick-17b-128e-instruct
Original result ↗1,127.804
- source id
- lmarena
- leaderboard name
- LMArena Vision Style Control Creative Writing
- dimension
- multimodal
- metric
- Arena rating
- rank
- 33
- sample size
- 250
- evaluated at
- 2026-01-09T00:00:00.000Z
- source model id
- llama-4-maverick-17b-128e-instruct
- confidence lower
- 1090.048890623918
- confidence upper
- 1165.559030360699
Original result ↗1,112.913
- source id
- lmarena
- leaderboard name
- LMArena Vision Creative Writing
- dimension
- multimodal
- metric
- Arena rating
- rank
- 34
- sample size
- 250
- evaluated at
- 2026-01-09T00:00:00.000Z
- source model id
- llama-4-maverick-17b-128e-instruct
- confidence lower
- 1074.9475947680562
- confidence upper
- 1150.8782117469716
Original result ↗69.1%
- source id
- openCompass
- leaderboard name
- GPQA-Diamond
- dimension
- general
- metric
- GPQA-Diamond
- rank
- 35
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- GPQA-Diamond
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Llama4-Maverick-17B-128E-Instruct
Original result ↗87.1%
- source id
- openCompass
- leaderboard name
- IFEval
- dimension
- general
- metric
- IFEval
- rank
- 36
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- IFEval
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Llama4-Maverick-17B-128E-Instruct
Original result ↗47.216%
- source id
- liveBench
- leaderboard name
- LiveBench Language
- dimension
- intelligence
- metric
- Average score
- rank
- 37
- sample size
- 240
- evaluated at
- 2025-04-07T16:07:43.128Z
- source model id
- llama-4-maverick-17b-128e-instruct
Original result ↗51.111%
- source id
- liveBench
- leaderboard name
- LiveBench Coding
- dimension
- coding
- metric
- Average score
- rank
- 43
- sample size
- 90
- evaluated at
- 2025-04-06T00:20:15.928Z
- source model id
- llama-4-maverick-17b-128e-instruct
Original result ↗5.3%
- source id
- openCompass
- leaderboard name
- HLE
- dimension
- general
- metric
- HLE
- rank
- 45
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- HLE
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Llama4-Maverick-17B-128E-Instruct
Original result ↗48.2%
- source id
- openCompass
- leaderboard name
- Average
- dimension
- general
- metric
- Average
- rank
- 46
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- Average
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Llama4-Maverick-17B-128E-Instruct
Original result ↗33.2%
- source id
- openCompass
- leaderboard name
- LiveCodeBenchV6
- dimension
- general
- metric
- LiveCodeBenchV6
- rank
- 47
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- LiveCodeBenchV6
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Llama4-Maverick-17B-128E-Instruct
Original result ↗Published prices
1–3 / 3
$0.17
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.35
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$1.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
Provider specifications & access
1–2 / 2
- source model id
- publishers/hf-meta-llama/models/llama-4-maverick-17b-128e-instruct@001
- hosting
- vertex-self-deployed
- source model id
- publishers/meta/models/llama4@llama-4-maverick-17b-128e-instruct
- hosting
- vertex-self-deployed
Safety assessments
1–1 / 1
31 / 100
- source id
- stanfordFMTI
- canonical model id
- meta-llama/llama-4-maverick-17b-128e-instruct
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- Meta
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 31 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–3 / 3
- field
- context_length
- value
- 524288
- normalized value
- 524288
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-08-31
- normalized value
- 2024-08-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- openRouter
- confidence
- 1
Explore more models