Anthropic: Claude Sonnet 4.6

Sonnet 4.6 is Anthropic's most capable Sonnet-class model yet, with frontier performance across coding, agents, and professional work. It excels at iterative development, complex codebase navigation,…

anthropic/claude-4.6-sonnet-20260217

Sonnet 4.6 is Anthropic's most capable Sonnet-class model yet, with frontier performance across coding, agents, and professional work. It excels at iterative development, complex codebase navigation, end-to-end project management with memory, polished document creation, and confident computer use for web QA and workflow automation.

Specifications & capabilities

Input
text, image, file
Output
text
Context window · tokens
1000000
Knowledge cutoff
2025-08
Published model ID
claude-sonnet-4-6
Capabilities
batch, computerUse, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗

Evaluations

1–15 / 15

OpenRouter Benchmarks

74.2%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Claude Platform on AWS
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
9
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Claude Platform on AWS
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T13:12:27.229Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7420000999999999
accuracy_std_dev
0.028759447531028826
cost_per_task_usd
0.344831642
output_tokens_per_task
11813.65
latency_seconds_per_task
200.277256
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
12
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

74.578%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Amazon Bedrock
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
11
sample size
1500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
30
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Amazon Bedrock
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:39:28.528Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7457779
accuracy_std_dev
0.024569333893222824
cost_per_task_usd
0.3685412306666667
output_tokens_per_task
11703.939333333334
latency_seconds_per_task
148.47426666666667
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
31
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

71.962%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Claude Platform on AWS
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
12
sample size
2178
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Claude Platform on AWS
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:38:33.095Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2178
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7196204545454546
accuracy_std_dev
0.12503183552080838
cost_per_task_usd
0.6236153714416895
output_tokens_per_task
41402.94995408632
latency_seconds_per_task
646.0125633608815
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
12
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

74.467%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Azure
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
13
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Azure
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T13:31:12.841Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7446667
accuracy_std_dev
0.01956190515798501
cost_per_task_usd
0.3469682859999999
output_tokens_per_task
11882.416
latency_seconds_per_task
215.22382199999998
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
40
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

73.8%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Anthropic
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
13
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Anthropic
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T13:36:32.496Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.738
accuracy_std_dev
0.020232086728758366
cost_per_task_usd
0.345152742
output_tokens_per_task
11819.48
latency_seconds_per_task
207.631326
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
16
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

71.503%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Anthropic
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
16
sample size
2178
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Anthropic
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:22:47.508Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2178
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7150290909090908
accuracy_std_dev
0.14315254393475418
cost_per_task_usd
0.6259749265381084
output_tokens_per_task
41568.741965105604
latency_seconds_per_task
641.693616620753
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
16
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

73.6%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Google
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
18
sample size
1500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
30
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Google
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T13:17:20.726Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7359999666666666
accuracy_std_dev
0.028627806219924643
cost_per_task_usd
0.3641733786666666
output_tokens_per_task
11882.770666666667
latency_seconds_per_task
196.84098666666665
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
36
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

72.054%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Amazon Bedrock
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
30
sample size
6534
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
33
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Amazon Bedrock
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:31:32.893Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
6534
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7205387272727272
accuracy_std_dev
0.13030117338555663
cost_per_task_usd
0.6590140376492195
output_tokens_per_task
41017.19651056015
latency_seconds_per_task
429.14430731558
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
32
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

71.97%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Google
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
33
sample size
6531
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
33
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Google
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:51:32.081Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
6531
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.719695823610473
accuracy_std_dev
0.13461887126264796
cost_per_task_usd
0.6638264376052672
output_tokens_per_task
41332.92175777063
latency_seconds_per_task
604.2676686571735
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
40
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

72.054%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Azure
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
36
sample size
2178
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Azure
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:28:17.671Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2178
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7205387272727274
accuracy_std_dev
0.14017884451338383
cost_per_task_usd
0.6223622332415059
output_tokens_per_task
41343.55968778696
latency_seconds_per_task
640.560214876033
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
43
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

74%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · auto-routing
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
38
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:07:21.658Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.74
accuracy_std_dev
0.027325104453597256
cost_per_task_usd
0.328687276
output_tokens_per_task
11587.26
latency_seconds_per_task
196.12096400000001
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
128
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

OpenRouter Benchmarks

72.084%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · auto-routing
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
99
sample size
2178
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:09:30.979Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2178
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.720845
accuracy_std_dev
0.13313462084603755
cost_per_task_usd
0.6171295247933883
output_tokens_per_task
40986.32323232323
latency_seconds_per_task
640.7006698806244
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
132
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-4.6-sonnet-20260217
source model id
anthropic/claude-4.6-sonnet-20260217
Original result ↗

Published prices

61–68 / 68

anthropic/claude-sonnet-4.6 ↗

$4.13

Inference provider
vercelAIGatewayCatalog
Platform
vertexAnthropic
Unit
cache_write_tokens_per_million
Currency
USD
Billing conditions
region
scope
specific
geo_region
us
provider_region
us-east5
endpoint
vertexAnthropic | anthropic/claude-sonnet-4.6
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cache write / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-sonnet-4.6 ↗

$3.00

Inference provider
vercelAIGatewayCatalog
Platform
vertexAnthropic
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
vertexAnthropic | anthropic/claude-sonnet-4.6
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-sonnet-4.6 ↗

$3.30

Inference provider
vercelAIGatewayCatalog
Platform
vertexAnthropic
Unit
input_tokens_per_million
Currency
USD
Billing conditions
region
scope
specific
geo_region
eu
provider_region
europe-west1
endpoint
vertexAnthropic | anthropic/claude-sonnet-4.6
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-sonnet-4.6 ↗

$3.30

Inference provider
vercelAIGatewayCatalog
Platform
vertexAnthropic
Unit
input_tokens_per_million
Currency
USD
Billing conditions
region
scope
specific
geo_region
us
provider_region
us-east5
endpoint
vertexAnthropic | anthropic/claude-sonnet-4.6
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-sonnet-4.6 ↗

$15.00

Inference provider
vercelAIGatewayCatalog
Platform
vertexAnthropic
Unit
output_tokens_per_million
Currency
USD
Billing conditions
endpoint
vertexAnthropic | anthropic/claude-sonnet-4.6
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-sonnet-4.6 ↗

$16.50

Inference provider
vercelAIGatewayCatalog
Platform
vertexAnthropic
Unit
output_tokens_per_million
Currency
USD
Billing conditions
region
scope
specific
geo_region
eu
provider_region
europe-west1
endpoint
vertexAnthropic | anthropic/claude-sonnet-4.6
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-sonnet-4.6 ↗

$16.50

Inference provider
vercelAIGatewayCatalog
Platform
vertexAnthropic
Unit
output_tokens_per_million
Currency
USD
Billing conditions
region
scope
specific
geo_region
us
provider_region
us-east5
endpoint
vertexAnthropic | anthropic/claude-sonnet-4.6
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-sonnet-4.6 ↗

$0.01

Inference provider
vercelAIGatewayCatalog
Platform
vertexAnthropic
Unit
request
Currency
USD
Billing conditions
charge
web_search
endpoint
vertexAnthropic | anthropic/claude-sonnet-4.6
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Web search
Pricing via
Vercel AI Gateway

Provider specifications & access

1–5 / 5

Claude Sonnet 4.6 ↗

source model id
anthropic.claude-sonnet-4-6
hosting
bedrock-managed
platforms
  • Amazon Bedrock
architecture
input_modalities
  • text
  • image
output_modalities
  • text
capabilities

    Claude Sonnet 4.6 ↗

    source model id
    anthropic/claude-sonnet-4.6
    hosting
    third-party
    platforms
    • Anthropic
    context length
    200000
    max output tokens
    128000
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities

      Claude Sonnet 4.6 ↗

      source model id
      anthropic/claude-sonnet-4.6
      hosting
      vercel-gateway
      platforms
      • Vercel AI Gateway
      context length
      1000000
      max output tokens
      128000
      architecture
      input_modalities
      • text
      • image
      • pdf
      output_modalities
      • text
      capabilities
      • promptCaching
      • reasoning
      • structuredOutputs
      • toolCalling
      • visionInput
      • webSearch
      regions
      • eu
      • us

      Claude Sonnet 4.6 · azureml-anthropic v1 ↗

      source model id
      azureml-anthropic/claude-sonnet-4-6@1
      hosting
      foundry-managed-compute
      platforms
      • Microsoft Foundry
      context length
      1000000
      max output tokens
      128000
      architecture
      input_modalities
      • text
      • image
      • code
      output_modalities
      • text
      capabilities
      • reasoning
      • toolCalling

      Claude Sonnet 4.6 on Google Cloud ↗

      source model id
      publishers/anthropic/models/claude-sonnet-4-6@default
      hosting
      vertex-managed
      platforms
      • Google Cloud Vertex AI
      max input tokens
      1000000
      max output tokens
      128000
      architecture
      input_modalities
      • file
      • image
      • text
      output_modalities
      • text
      capabilities
      • batch
      • computerUse
      • multimodal
      • promptCaching
      • reasoning
      • toolCalling
      • visionInput
      • webSearch

      Safety assessments

      1–1 / 1

      Foundation Model Transparency Index 2025 ↗

      46 / 100

      source id
      stanfordFMTI
      canonical model id
      anthropic/claude-4.6-sonnet-20260217
      taxonomy
      transparency
      hazard category
      transparency
      hazard name
      Organization transparency
      policy area
      Anthropic
      assessment type
      organization-level disclosure
      metric
      disclosure_score
      result label
      46 / 100
      sample size
      100
      updated at
      2026-11-09T18:08:05.227Z

      Source specification observations

      1–14 / 14

      azureFoundry ↗

      field
      context_length
      value
      1000000
      normalized value
      1000000
      scope
      source-model
      source id
      azureFoundry
      confidence
      1

      cloudflareWorkersAI ↗

      field
      context_length
      value
      200000
      normalized value
      200000
      scope
      source-model
      source id
      cloudflareWorkersAI
      confidence
      1

      openRouter ↗

      field
      context_length
      value
      1000000
      normalized value
      1000000
      scope
      source-model
      source id
      openRouter
      confidence
      1

      openRouter ↗

      field
      context_length
      value
      1000000
      normalized value
      1000000
      scope
      source-model
      source id
      openRouter
      confidence
      1

      vercelAIGatewayCatalog ↗

      field
      context_length
      value
      1000000
      normalized value
      1000000
      scope
      source-model
      source id
      vercelAIGatewayCatalog
      confidence
      1

      vercelAIGatewayCatalog ↗

      field
      knowledge_cutoff
      reported field
      knowledge_cutoff
      value
      2025-08-31
      normalized value
      2025-08-31
      precision
      day
      scope
      source-model
      source id
      vercelAIGatewayCatalog
      confidence
      1

      vertexAI ↗

      field
      max_input_tokens
      value
      1000000
      normalized value
      1000000
      scope
      source-model
      source id
      vertexAI
      confidence
      1

      azureFoundry ↗

      field
      max_output_tokens
      value
      128000
      normalized value
      128000
      scope
      source-model
      source id
      azureFoundry
      confidence
      1

      cloudflareWorkersAI ↗

      field
      max_output_tokens
      value
      128000
      normalized value
      128000
      scope
      source-model
      source id
      cloudflareWorkersAI
      confidence
      1

      openRouter ↗

      field
      max_output_tokens
      value
      128000
      normalized value
      128000
      scope
      source-model
      source id
      openRouter
      confidence
      1

      openRouter ↗

      field
      max_output_tokens
      value
      128000
      normalized value
      128000
      scope
      source-model
      source id
      openRouter
      confidence
      1

      vercelAIGatewayCatalog ↗

      field
      max_output_tokens
      value
      128000
      normalized value
      128000
      scope
      source-model
      source id
      vercelAIGatewayCatalog
      confidence
      1

      vertexAI ↗

      field
      max_output_tokens
      value
      128000
      normalized value
      128000
      scope
      source-model
      source id
      vertexAI
      confidence
      1

      azureFoundry ↗

      field
      knowledge_cutoff
      reported field
      training_data_cutoff
      value
      August 2025
      normalized value
      2025-08
      precision
      month
      scope
      source-model
      source id
      azureFoundry
      confidence
      1

      Original sources

      1–20 / 20

      Anthropic · Model reference ↗

      Google Cloud · Model page ↗

      Microsoft Foundry · Model page ↗

      OpenRouter · Model page ↗

      OpenRouter · Model page ↗

      Vercel AI Gateway · Model page ↗

      Google Cloud · Documentation ↗

      Microsoft Foundry · Model documentation ↗

      GitHub · Source snapshot ↗

      Amazon Web Services · Model page ↗

      Amazon Bedrock · Model documentation ↗

      Amazon Web Services · Catalog ↗

      Cloudflare AI · Catalog ↗

      Epoch AI · Catalog ↗

      GitHub · Catalog ↗

      Google Cloud · Catalog ↗

      Microsoft Foundry · Catalog ↗

      OpenRouter · Catalog ↗

      OpenRouter · Catalog ↗

      Vercel AI Gateway · Catalog ↗

      Source links

      1–10 / 10

      Explore more models

      Models

      Loading
      EVIDENCE / COMPARISON

      Benchmarks

      Explore every benchmark

      Loading

      These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

      Install TensorTrace