Meta: Muse Glimmer 30B
Muse Glimmer 30B is a dense, open-weight multimodal model from Meta Superintelligence Labs, distilled from Muse Spark and optimized for autonomous agents on consumer hardware. It is suited for long-h…
meta/muse-glimmer-30b-20260810
Muse Glimmer 30B is a dense, open-weight multimodal model from Meta Superintelligence Labs, distilled from Muse Spark and optimized for autonomous agents on consumer hardware. It is suited for long-horizon agentic and coding workflows, with multi-step reasoning, reliable tool use, failure recovery, image understanding, and multilingual support across more than 100 languages.
Specifications & capabilities
- Input
- text, image
- Output
- text
- Context window · tokens
- 131072
- Capabilities
- multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput
Description source ↗Evaluations
1–15 / 15
77.069%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Parasail
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 3
- sample size
- 149
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Parasail
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-08-29T08:48:12.957Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 149
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7706935637583894
- accuracy_std_dev
- 0.013644816256244217
- cost_per_task_usd
- 0.09587820805369127
- output_tokens_per_task
- 13114.979865771813
- latency_seconds_per_task
- 288.02166442953023
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 28
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗77.571%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · DeepInfra
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 4
- sample size
- 350
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- DeepInfra
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-08T22:42:47.098Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 350
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7757142857142857
- accuracy_std_dev
- 0.033557166846673904
- cost_per_task_usd
- 0.03545617428571429
- output_tokens_per_task
- 13844.954285714286
- latency_seconds_per_task
- 502.38840285714286
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 63
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗76.111%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Phala
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 5
- sample size
- 300
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Phala
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-03T08:20:09.018Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 300
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7611111666666667
- accuracy_std_dev
- 0.013005103465135887
- cost_per_task_usd
- 0.033468996666666674
- output_tokens_per_task
- 13650.26
- latency_seconds_per_task
- 272.35700333333335
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 23
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗74.667%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Fireworks
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 5
- sample size
- 200
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 4
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Fireworks
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-09-19T08:01:36.427Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 200
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7466665
- accuracy_std_dev
- 0.01054097823970812
- cost_per_task_usd
- 0.047252215
- output_tokens_per_task
- 13256.62
- latency_seconds_per_task
- 265.241545
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 14
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗75%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Together
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 7
- sample size
- 300
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Together
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-03T08:33:09.729Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 300
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.75
- accuracy_std_dev
- 0.01666660000320003
- cost_per_task_usd
- 0.041500640000000005
- output_tokens_per_task
- 12674.073333333334
- latency_seconds_per_task
- 199.60765333333333
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 22
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗81.178%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Fireworks
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 15
- sample size
- 990
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Fireworks
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-09-19T09:05:52.633Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 990
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8117845999999999
- accuracy_std_dev
- 0.01594972366657178
- cost_per_task_usd
- 0.026062873737373735
- output_tokens_per_task
- 17452.424242424244
- latency_seconds_per_task
- 323.33302424242424
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 17
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗82.074%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Phala
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 17
- sample size
- 1376
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Phala
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-03T09:25:02.390Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1376
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.820736340116279
- accuracy_std_dev
- 0.02413812698210204
- cost_per_task_usd
- 0.01966482122093023
- output_tokens_per_task
- 17874.627180232557
- latency_seconds_per_task
- 283.5174375
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 24
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗82.121%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Parasail
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 21
- sample size
- 591
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Parasail
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-08-29T09:59:35.603Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 591
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8212070710659898
- accuracy_std_dev
- 0.011946281848340899
- cost_per_task_usd
- 0.01807649915397631
- output_tokens_per_task
- 16413.382402707277
- latency_seconds_per_task
- 302.9400592216582
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 36
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗81.866%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Together
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 22
- sample size
- 1386
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Together
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-03T10:10:35.343Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1386
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8186628571428571
- accuracy_std_dev
- 0.011047136143797251
- cost_per_task_usd
- 0.024918836219336223
- output_tokens_per_task
- 16519.824675324675
- latency_seconds_per_task
- 190.47732106782107
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 25
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗75.667%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · auto-routing
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 23
- sample size
- 300
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- auto-routing
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-03T08:15:57.213Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 300
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7566665
- accuracy_std_dev
- 0.0251662251622421
- cost_per_task_usd
- 0.03758577333333333
- output_tokens_per_task
- 13605.943333333333
- latency_seconds_per_task
- 199.1407
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 128
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗79.23%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · DeepInfra
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 30
- sample size
- 1572
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 8
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- DeepInfra
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-08T22:29:11.504Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1572
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.7923028091603053
- accuracy_std_dev
- 0.012838036519947284
- cost_per_task_usd
- 0.01830340776081425
- output_tokens_per_task
- 15102.418575063613
- latency_seconds_per_task
- 189.91377798982188
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 71
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗80.784%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · auto-routing
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 79
- sample size
- 1386
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- auto-routing
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-03T09:13:36.277Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1386
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8078402857142857
- accuracy_std_dev
- 0.013630118652814278
- cost_per_task_usd
- 0.02220728354978355
- output_tokens_per_task
- 17520.37012987013
- latency_seconds_per_task
- 246.89744444444443
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 132
- benchmark_evaluation
- true
- openrouter_permaslug
- meta/muse-glimmer-30b-20260810
- source model id
- meta/muse-glimmer-30b-20260810
Original result ↗49
- source id
- artificialAnalysis
- dimension
- coding
- metric
- Coding Index
Original result ↗8.5
- source id
- artificialAnalysis
- dimension
- agentic
- metric
- Agentic Index
Original result ↗6.3
- source id
- artificialAnalysis
- dimension
- intelligence
- metric
- Intelligence Index
Original result ↗Published prices
1–6 / 6
$0.04
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.30
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$1.20
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.03
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.24
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.96
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
Provider specifications & access
1–1 / 1
- source model id
- meta/muse-glimmer-30b
- hosting
- nvidia-catalog
- context length
- 131072
Safety assessments
1–1 / 1
31 / 100
- source id
- stanfordFMTI
- canonical model id
- meta/muse-glimmer-30b-20260810
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- Meta
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 31 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–5 / 5
- field
- context_length
- value
- 131072
- normalized value
- 131072
- scope
- source-model
- source id
- nvidiaNim
- confidence
- 1
- field
- context_length
- value
- 131072
- normalized value
- 131072
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 131072
- normalized value
- 131072
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 16384
- normalized value
- 16384
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 16384
- normalized value
- 16384
- scope
- source-model
- source id
- openRouter
- confidence
- 1
Explore more models