Meta: Muse Spark Open

Muse Spark Open is a coding-optimized open-weight model. Trained specifically for agentic coding workflows, it delivers higher first-attempt accuracy, reliable tool calling, and deep codebase underst…

meta/muse-spark-open-20261015

Muse Spark Open is a coding-optimized open-weight model. Trained specifically for agentic coding workflows, it delivers higher first-attempt accuracy, reliable tool calling, and deep codebase understanding across the repos developers actually work in.

Specifications & capabilities

Input
text, image, video, file, audio
Output
text
Context window · tokens
1048576
Capabilities
audioInput, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput
Description source ↗

Evaluations

1–15 / 15

OpenRouter Benchmarks

90.152%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · BaseTen
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
2
sample size
396
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
BaseTen
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:17:04.260Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
396
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.901515
accuracy_std_dev
0.0025249999999999995
cost_per_task_usd
0.0720575808080808
output_tokens_per_task
16730.29797979798
latency_seconds_per_task
100.99045707070707
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
16
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

75.51%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Modal
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
2
sample size
98
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Modal
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:36:21.331Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
98
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7551024285714285
accuracy_std_dev
0.011805499999999969
cost_per_task_usd
0.27501580612244897
output_tokens_per_task
8842.591836734693
latency_seconds_per_task
65.99772448979591
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
11
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

90.963%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Modal
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
3
sample size
308
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Modal
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:15:03.291Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
308
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.909632
accuracy_std_dev
0.0055275000000000185
cost_per_task_usd
0.05270355519480519
output_tokens_per_task
12143.461038961039
latency_seconds_per_task
55.28537662337662
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
11
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

90.486%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Makora
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
3
sample size
939
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Makora
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:13:57.643Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
939
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.9048634068157615
accuracy_std_dev
0.012266915351464668
cost_per_task_usd
0.06399653567625134
output_tokens_per_task
14917.831735889244
latency_seconds_per_task
75.58011395101173
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
12
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

90.236%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Fireworks
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
4
sample size
990
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Fireworks
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:52:05.708Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
990
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.9023568
accuracy_std_dev
0.003838970664123373
cost_per_task_usd
0.0695745484848485
output_tokens_per_task
16158.315151515151
latency_seconds_per_task
344.2386484848485
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
17
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

75.168%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Nebius
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
5
sample size
149
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:50:00.858Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
149
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7516778724832214
accuracy_std_dev
0.030780262062995275
cost_per_task_usd
0.08526638255033557
output_tokens_per_task
8282.208053691274
latency_seconds_per_task
172.05979865771812
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
23
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

89.731%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Together
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
6
sample size
396
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Together
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:49:18.270Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
396
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8973059999999999
accuracy_std_dev
0.0033669999999999534
cost_per_task_usd
0.07477493686868687
output_tokens_per_task
17437.89393939394
latency_seconds_per_task
142.684398989899
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

89.703%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Nebius
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
6
sample size
594
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:27:47.368Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
594
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8970256666666665
accuracy_std_dev
0.00667522615513675
cost_per_task_usd
0.06776180639730639
output_tokens_per_task
15778.796296296296
latency_seconds_per_task
222.6862255892256
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

74.133%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Fireworks
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
6
sample size
250
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Fireworks
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T14:40:25.043Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
250
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7413334
accuracy_std_dev
0.02436751571744648
cost_per_task_usd
0.097602524
output_tokens_per_task
9920.02
latency_seconds_per_task
217.80416
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
14
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

72.336%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Makora
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
8
sample size
244
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Makora
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:46:18.075Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
244
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.723360643442623
accuracy_std_dev
0.019132270021092652
cost_per_task_usd
0.0756424631147541
output_tokens_per_task
8337.151639344262
latency_seconds_per_task
76.63130327868853
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
11
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

72.281%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · BaseTen
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
9
sample size
95
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
BaseTen
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:12:21.006Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
95
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7228067157894738
accuracy_std_dev
0.002917999999999976
cost_per_task_usd
0.06760965263157895
output_tokens_per_task
5886.4315789473685
latency_seconds_per_task
49.83126315789474
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
14
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

73.333%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Together
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
12
sample size
100
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Together
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:29:29.503Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
100
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7333335
accuracy_std_dev
0.00666650000000002
cost_per_task_usd
0.08901925000000001
output_tokens_per_task
8634.62
latency_seconds_per_task
93.61232000000001
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
22
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

90.152%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · auto-routing
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
20
sample size
396
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:09:26.224Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
396
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.9015150000000001
accuracy_std_dev
0.004208999999999963
cost_per_task_usd
0.0736977297979798
output_tokens_per_task
17125.45707070707
latency_seconds_per_task
138.58901262626262
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
132
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

OpenRouter Benchmarks

75.667%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · auto-routing
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
22
sample size
100
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:44:24.100Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
100
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7566669999999999
accuracy_std_dev
0.010000000000000009
cost_per_task_usd
0.08839812
output_tokens_per_task
8719.83
latency_seconds_per_task
73.38011
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
128
benchmark_evaluation
true
openrouter_permaslug
meta/muse-spark-open-20261015
source model id
meta/muse-spark-open-20261015
Original result ↗

Published prices

1–6 / 6

meta/muse-spark-open ↗

$0.15

Inference provider
cloudflareWorkersAI
Platform
Fireworks AI
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
Billing label
Cached input tokens (per 1M)
Pricing via
Cloudflare AI Gateway

meta/muse-spark-open ↗

$1.25

Inference provider
cloudflareWorkersAI
Platform
Fireworks AI
Unit
input_tokens_per_million
Currency
USD
Billing conditions
Billing label
Input tokens (per 1M)
Pricing via
Cloudflare AI Gateway

meta/muse-spark-open ↗

$4.25

Inference provider
cloudflareWorkersAI
Platform
Fireworks AI
Unit
output_tokens_per_million
Currency
USD
Billing conditions
Billing label
Output tokens (per 1M)
Pricing via
Cloudflare AI Gateway

meta/muse-spark-open ↗

$0.15

Inference provider
openRouter
Platform
OpenRouter
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions

meta/muse-spark-open ↗

$1.25

Inference provider
openRouter
Platform
OpenRouter
Unit
input_tokens_per_million
Currency
USD
Billing conditions

meta/muse-spark-open ↗

$4.25

Inference provider
openRouter
Platform
OpenRouter
Unit
output_tokens_per_million
Currency
USD
Billing conditions

Provider specifications & access

1–1 / 1

Muse Spark Open ↗

source model id
meta/muse-spark-open
hosting
third-party
platforms
  • Fireworks AI
context length
1040000
architecture
input_modalities
  • text
  • audio
output_modalities
  • text
capabilities
  • toolCalling

Safety assessments

1–1 / 1

Foundation Model Transparency Index 2025 ↗

31 / 100

source id
stanfordFMTI
canonical model id
meta/muse-spark-open-20261015
taxonomy
transparency
hazard category
transparency
hazard name
Organization transparency
policy area
Meta
assessment type
organization-level disclosure
metric
disclosure_score
result label
31 / 100
sample size
100
updated at
2026-11-09T18:08:05.227Z

Source specification observations

1–3 / 3

cloudflareWorkersAI ↗

field
context_length
value
1040000
normalized value
1040000
scope
source-model
source id
cloudflareWorkersAI
confidence
1

openRouter ↗

field
context_length
value
1048576
normalized value
1048576
scope
source-model
source id
openRouter
confidence
1

openRouter ↗

field
max_output_tokens
value
262144
normalized value
262144
scope
source-model
source id
openRouter
confidence
1

Original sources

1–9 / 9

Hugging Face · meta-models/Muse-Spark-Open-816B-A42B ↗

OpenRouter · Model page ↗

GitHub · Source snapshot ↗

Cloudflare AI · Catalog ↗

Epoch AI · Catalog ↗

GitHub · Catalog ↗

Hugging Face · Catalog ↗

OpenRouter · Catalog ↗

OpenRouter · Catalog ↗

Source links

1–8 / 8

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace