Anthropic: Claude Opus 4.8

Claude Opus 4.8 is Anthropic's most capable generally available model in the Opus family. It supports text, image, and file inputs with text output, with reasoning support and a 1M-token context wind…

anthropic/claude-4.8-opus-20260528

Claude Opus 4.8 is Anthropic's most capable generally available model in the Opus family. It supports text, image, and file inputs with text output, with reasoning support and a 1M-token context window. It is suited for highly autonomous agents, long-horizon agentic work, knowledge work, and memory-driven tasks where coherence over extended sessions matters.

It is particularly strong on multi-step reasoning, complex coding, and end-to-end project orchestration - large codebases, multi-stage debugging, and long-running asynchronous agent pipelines. Beyond coding, it handles knowledge work such as drafting documents, building presentations, and analyzing data, maintaining quality across very long outputs.

Specifications & capabilities

Input
text, image, file
Output
text
Context window · tokens
1000000
Knowledge cutoff
2026-01
Published model ID
claude-opus-4-8
Capabilities
batch, computerUse, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗

Evaluations

1–15 / 15

OpenRouter Benchmarks

89.96%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Claude Platform on AWS
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
2
sample size
2178
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Claude Platform on AWS
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:27:45.200Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2178
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.899602
accuracy_std_dev
0.006144750235326509
cost_per_task_usd
0.20230806932966028
output_tokens_per_task
7867.12075298439
latency_seconds_per_task
96.2744453627181
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
12
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

77.067%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Claude Platform on AWS
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
3
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Claude Platform on AWS
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:58:26.584Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7706667
accuracy_std_dev
0.021949532013462166
cost_per_task_usd
0.48758016800000004
output_tokens_per_task
7988.41
latency_seconds_per_task
123.17747
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
12
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

89.669%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Anthropic
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
4
sample size
2178
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Anthropic
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:27:47.283Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2178
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.896694090909091
accuracy_std_dev
0.006667655625269636
cost_per_task_usd
0.208940303030303
output_tokens_per_task
8143.670798898072
latency_seconds_per_task
100.74017171717172
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
16
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

76.444%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Amazon Bedrock
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
4
sample size
1500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
30
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Amazon Bedrock
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:26:06.842Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7644444333333335
accuracy_std_dev
0.01919359203950342
cost_per_task_usd
0.5300209846666668
output_tokens_per_task
8055.547333333333
latency_seconds_per_task
117.52525933333334
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
31
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

77.067%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Anthropic
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
5
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Anthropic
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:50:39.610Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7706667
accuracy_std_dev
0.025854399153915776
cost_per_task_usd
0.5051475320000001
output_tokens_per_task
7988.984
latency_seconds_per_task
119.86874800000001
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
16
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

76.667%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Azure
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
6
sample size
1000
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
20
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Azure
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T13:09:20.558Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1000
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7666667
accuracy_std_dev
0.02658317763943958
cost_per_task_usd
0.5167896780000001
output_tokens_per_task
7924.367
latency_seconds_per_task
124.380271
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
40
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

76.733%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Google
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
7
sample size
1500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
30
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Google
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:31:31.431Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7673332999999999
accuracy_std_dev
0.02871832890118551
cost_per_task_usd
0.5323030653333332
output_tokens_per_task
8030.952666666667
latency_seconds_per_task
111.14459533333333
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
36
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

90.016%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Google
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
10
sample size
6534
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
33
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Google
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:31:12.840Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
6534
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.9001631212121214
accuracy_std_dev
0.005552397340584662
cost_per_task_usd
0.22268922956841136
output_tokens_per_task
8126.572849709213
latency_seconds_per_task
93.99823614937252
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
40
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

90.151%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Azure
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
12
sample size
4356
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
22
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Azure
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:46:46.997Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
4356
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.9015149999999998
accuracy_std_dev
0.006314360868830512
cost_per_task_usd
0.20703088957759413
output_tokens_per_task
7837.382690541782
latency_seconds_per_task
97.68647543617999
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
43
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

90.042%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Amazon Bedrock
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
12
sample size
6534
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
33
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Amazon Bedrock
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:23:44.152Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
6534
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.900418212121212
accuracy_std_dev
0.006022033356174897
cost_per_task_usd
0.21616242209978576
output_tokens_per_task
7884.330272421182
latency_seconds_per_task
92.58719115396389
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
32
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

76.8%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · auto-routing
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
16
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:30:31.663Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7680001999999999
accuracy_std_dev
0.021457811839980336
cost_per_task_usd
0.49101839
output_tokens_per_task
8037.726
latency_seconds_per_task
123.06560400000001
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
128
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

OpenRouter Benchmarks

89.715%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · auto-routing
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
24
sample size
2178
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:32:06.814Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2178
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8971531818181817
accuracy_std_dev
0.011450877590966798
cost_per_task_usd
0.20187068044077136
output_tokens_per_task
7855.07943067034
latency_seconds_per_task
96.5283347107438
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
132
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.8-opus-20260528
source model id
anthropic/claude-4.8-opus-20260528
Original result ↗

Published prices

21–40 / 77

anthropic/claude-opus-4.8 ↗

$5.00

Inference provider
vercelAIGatewayCatalog
Platform
anthropic
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
anthropic | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$5.50

Inference provider
vercelAIGatewayCatalog
Platform
anthropic
Unit
input_tokens_per_million
Currency
USD
Billing conditions
region
scope
zone
geo_region
us
endpoint
anthropic | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$25.00

Inference provider
vercelAIGatewayCatalog
Platform
anthropic
Unit
output_tokens_per_million
Currency
USD
Billing conditions
endpoint
anthropic | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$27.50

Inference provider
vercelAIGatewayCatalog
Platform
anthropic
Unit
output_tokens_per_million
Currency
USD
Billing conditions
region
scope
zone
geo_region
us
endpoint
anthropic | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$0.01

Inference provider
vercelAIGatewayCatalog
Platform
anthropic
Unit
request
Currency
USD
Billing conditions
charge
web_search
endpoint
anthropic | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Web search
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$0.50

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$0.55

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
region
scope
specific
geo_region
eu
provider_region
eu-central-1
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$0.55

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
region
scope
zone
geo_region
us
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$6.25

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
cache_write_tokens_per_million
Currency
USD
Billing conditions
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cache write / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$6.88

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
cache_write_tokens_per_million
Currency
USD
Billing conditions
region
scope
specific
geo_region
eu
provider_region
eu-central-1
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cache write / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$6.88

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
cache_write_tokens_per_million
Currency
USD
Billing conditions
region
scope
zone
geo_region
us
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cache write / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$5.00

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$5.50

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
input_tokens_per_million
Currency
USD
Billing conditions
region
scope
specific
geo_region
eu
provider_region
eu-central-1
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$5.50

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
input_tokens_per_million
Currency
USD
Billing conditions
region
scope
zone
geo_region
us
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$25.00

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
output_tokens_per_million
Currency
USD
Billing conditions
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$27.50

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
output_tokens_per_million
Currency
USD
Billing conditions
region
scope
specific
geo_region
eu
provider_region
eu-central-1
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$27.50

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
output_tokens_per_million
Currency
USD
Billing conditions
region
scope
zone
geo_region
us
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$0.01

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
request
Currency
USD
Billing conditions
charge
web_search
endpoint
bedrock | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Web search
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$0.50

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
endpoint
claudeaws | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-opus-4.8 ↗

$6.25

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
cache_write_tokens_per_million
Currency
USD
Billing conditions
endpoint
claudeaws | anthropic/claude-opus-4.8
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cache write / 1M
Pricing via
Vercel AI Gateway

Provider specifications & access

1–6 / 6

Claude Opus 4.8 ↗

source model id
anthropic.claude-opus-4-8
hosting
bedrock-managed
platforms
  • Amazon Bedrock
architecture
input_modalities
  • text
  • image
output_modalities
  • text
capabilities

    Claude Opus 4.8 ↗

    source model id
    anthropic/claude-opus-4.8
    hosting
    third-party
    platforms
    • Anthropic
    context length
    1000000
    max output tokens
    128000
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities
    • reasoning

    Claude Opus 4.8 ↗

    source model id
    anthropic/claude-opus-4.8
    hosting
    vercel-gateway
    platforms
    • Vercel AI Gateway
    context length
    1000000
    max output tokens
    128000
    architecture
    input_modalities
    • text
    • image
    • pdf
    output_modalities
    • text
    capabilities
    • promptCaching
    • reasoning
    • structuredOutputs
    • toolCalling
    • visionInput
    • webSearch
    regions
    • eu
    • us

    Claude Opus 4.8 · azureml-anthropic v1 ↗

    source model id
    azureml-anthropic/claude-opus-4-8@1
    hosting
    foundry-managed-compute
    platforms
    • Microsoft Foundry
    context length
    1000000
    max output tokens
    128000
    architecture
    input_modalities
    • text
    • image
    • code
    output_modalities
    • text
    capabilities
    • reasoning
    • toolCalling

    Claude Opus 4.8 · azureml-anthropic v2 ↗

    source model id
    azureml-anthropic/claude-opus-4-8@2
    hosting
    foundry-managed-compute
    platforms
    • Microsoft Foundry
    context length
    1000000
    max output tokens
    128000
    architecture
    input_modalities
    • text
    • image
    • code
    output_modalities
    • text
    capabilities
    • reasoning
    • toolCalling

    Claude Opus 4.8 on Google Cloud ↗

    source model id
    publishers/anthropic/models/claude-opus-4-8@default
    hosting
    vertex-managed
    platforms
    • Google Cloud Vertex AI
    max input tokens
    1000000
    max output tokens
    128000
    architecture
    input_modalities
    • file
    • image
    • text
    output_modalities
    • text
    capabilities
    • batch
    • computerUse
    • multimodal
    • promptCaching
    • reasoning
    • toolCalling
    • visionInput
    • webSearch

    Safety assessments

    1–1 / 1

    Foundation Model Transparency Index 2025 ↗

    46 / 100

    source id
    stanfordFMTI
    canonical model id
    anthropic/claude-4.8-opus-20260528
    taxonomy
    transparency
    hazard category
    transparency
    hazard name
    Organization transparency
    policy area
    Anthropic
    assessment type
    organization-level disclosure
    metric
    disclosure_score
    result label
    46 / 100
    sample size
    100
    updated at
    2026-11-09T18:08:05.227Z

    Source specification observations

    1–17 / 17

    azureFoundry ↗

    field
    context_length
    value
    1000000
    normalized value
    1000000
    scope
    source-model
    source id
    azureFoundry
    confidence
    1

    azureFoundry ↗

    field
    context_length
    value
    1000000
    normalized value
    1000000
    scope
    source-model
    source id
    azureFoundry
    confidence
    1

    cloudflareWorkersAI ↗

    field
    context_length
    value
    1000000
    normalized value
    1000000
    scope
    source-model
    source id
    cloudflareWorkersAI
    confidence
    1

    openRouter ↗

    field
    context_length
    value
    1000000
    normalized value
    1000000
    scope
    source-model
    source id
    openRouter
    confidence
    1

    openRouter ↗

    field
    context_length
    value
    1000000
    normalized value
    1000000
    scope
    source-model
    source id
    openRouter
    confidence
    1

    vercelAIGatewayCatalog ↗

    field
    context_length
    value
    1000000
    normalized value
    1000000
    scope
    source-model
    source id
    vercelAIGatewayCatalog
    confidence
    1

    vercelAIGatewayCatalog ↗

    field
    knowledge_cutoff
    reported field
    knowledge_cutoff
    value
    2026-01
    normalized value
    2026-01
    precision
    month
    scope
    source-model
    source id
    vercelAIGatewayCatalog
    confidence
    1

    vertexAI ↗

    field
    max_input_tokens
    value
    1000000
    normalized value
    1000000
    scope
    source-model
    source id
    vertexAI
    confidence
    1

    azureFoundry ↗

    field
    max_output_tokens
    value
    128000
    normalized value
    128000
    scope
    source-model
    source id
    azureFoundry
    confidence
    1

    azureFoundry ↗

    field
    max_output_tokens
    value
    128000
    normalized value
    128000
    scope
    source-model
    source id
    azureFoundry
    confidence
    1

    cloudflareWorkersAI ↗

    field
    max_output_tokens
    value
    128000
    normalized value
    128000
    scope
    source-model
    source id
    cloudflareWorkersAI
    confidence
    1

    openRouter ↗

    field
    max_output_tokens
    value
    128000
    normalized value
    128000
    scope
    source-model
    source id
    openRouter
    confidence
    1

    openRouter ↗

    field
    max_output_tokens
    value
    128000
    normalized value
    128000
    scope
    source-model
    source id
    openRouter
    confidence
    1

    vercelAIGatewayCatalog ↗

    field
    max_output_tokens
    value
    128000
    normalized value
    128000
    scope
    source-model
    source id
    vercelAIGatewayCatalog
    confidence
    1

    vertexAI ↗

    field
    max_output_tokens
    value
    128000
    normalized value
    128000
    scope
    source-model
    source id
    vertexAI
    confidence
    1

    azureFoundry ↗

    field
    knowledge_cutoff
    reported field
    training_data_cutoff
    value
    January 2026
    normalized value
    2026-01
    precision
    month
    scope
    source-model
    source id
    azureFoundry
    confidence
    1

    azureFoundry ↗

    field
    knowledge_cutoff
    reported field
    training_data_cutoff
    value
    January 2026
    normalized value
    2026-01
    precision
    month
    scope
    source-model
    source id
    azureFoundry
    confidence
    1

    Original sources

    1–20 / 21

    Anthropic · Model reference ↗

    Google Cloud · Model page ↗

    Microsoft Foundry · Model page ↗

    Microsoft Foundry · Model page ↗

    OpenRouter · Model page ↗

    OpenRouter · Model page ↗

    Vercel AI Gateway · Model page ↗

    Google Cloud · Documentation ↗

    Microsoft Foundry · Model documentation ↗

    GitHub · Source snapshot ↗

    Amazon Web Services · Model page ↗

    Amazon Bedrock · Model documentation ↗

    Amazon Web Services · Catalog ↗

    Cloudflare AI · Catalog ↗

    Epoch AI · Catalog ↗

    GitHub · Catalog ↗

    Google Cloud · Catalog ↗

    Microsoft Foundry · Catalog ↗

    OpenRouter · Catalog ↗

    OpenRouter · Catalog ↗

    Source links

    1–10 / 10

    Explore more models

    Models

    Loading
    EVIDENCE / COMPARISON

    Benchmarks

    Explore every benchmark

    Loading

    These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

    Install TensorTrace