Anthropic: Claude Haiku 5.5

Claude Haiku 5.5 is Anthropic's small, fast model for high-volume, cost-sensitive work such as summarization, subagents, and browser use. It succeeds Claude Haiku 4.5 with stronger coding, computer u…

anthropic/claude-haiku-5.5-20261007

Claude Haiku 5.5 is Anthropic's small, fast model for high-volume, cost-sensitive work such as summarization, subagents, and browser use. It succeeds Claude Haiku 4.5 with stronger coding, computer use, and knowledge work, and accepts text and image input with a 1M-token context window.

It is the first Haiku model with adjustable effort. Thinking is adaptive and on by default, with effort as the main lever for trading off depth, latency, and cost; it can be turned off at low, medium, and high effort.

Specifications & capabilities

Input
text, image, file
Output
text
Context window · tokens
1000000
Knowledge cutoff
2026-06
Published model ID
claude-haiku-5-5
Capabilities
fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗

Evaluations

1–14 / 14

OpenRouter Benchmarks

86.902%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Claude Platform on AWS
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
7
sample size
990
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Claude Platform on AWS
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:35:55.398Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
990
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8690236000000001
accuracy_std_dev
0.0056741455250989255
cost_per_task_usd
0.004051724242424242
output_tokens_per_task
7872.538383838384
latency_seconds_per_task
38.66653434343434
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
12
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

Vercel AI Gateway

4.804%

source id
vercelAIGateway
leaderboard name
AI Gateway Text Requests Share
dimension
popularity
metric
Requests
rank
7
evaluated at
2026-11-11T00:00:00.000Z
source model id
Claude Haiku 5.5
Original result ↗

OpenRouter Benchmarks

87.037%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Anthropic
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
10
sample size
990
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Anthropic
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:14:20.331Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
990
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8703704
accuracy_std_dev
0.00583193054142453
cost_per_task_usd
0.004018268686868687
output_tokens_per_task
7796.7444444444445
latency_seconds_per_task
37.45491717171717
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
16
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

OpenRouter Benchmarks

72.933%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Claude Platform on AWS
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
12
sample size
250
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Claude Platform on AWS
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:41:01.831Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
250
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7293334
accuracy_std_dev
0.026533159130416407
cost_per_task_usd
0.012403696
output_tokens_per_task
10764.472
latency_seconds_per_task
92.314648
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
12
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

OpenRouter Benchmarks

73.956%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Amazon Bedrock
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
14
sample size
750
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
15
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Amazon Bedrock
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:45:41.031Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
750
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7395555333333333
accuracy_std_dev
0.016947170784004716
cost_per_task_usd
0.013297554666666666
output_tokens_per_task
10746.964
latency_seconds_per_task
100.94712933333334
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
31
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

OpenRouter Benchmarks

73.333%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Anthropic
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
14
sample size
250
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Anthropic
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:35:21.238Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
250
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7333332
accuracy_std_dev
0.032659904062320846
cost_per_task_usd
0.012294704000000002
output_tokens_per_task
10698.828
latency_seconds_per_task
85.675876
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
16
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

OpenRouter Benchmarks

73.133%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Azure
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
16
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Azure
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:19:23.393Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7313332
accuracy_std_dev
0.020231941744676914
cost_per_task_usd
0.012560144
output_tokens_per_task
10525.228
latency_seconds_per_task
74.716332
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
40
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

OpenRouter Benchmarks

86.756%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Google
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
17
sample size
2970
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
15
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Google
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:38:41.553Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2970
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8675645333333333
accuracy_std_dev
0.00826267276262484
cost_per_task_usd
0.004299042424242424
output_tokens_per_task
7829.657575757576
latency_seconds_per_task
36.21724208754209
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
40
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

OpenRouter Benchmarks

73.6%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Google
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
19
sample size
750
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
15
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Google
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:42:11.484Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
750
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7359999333333334
accuracy_std_dev
0.0231964569463145
cost_per_task_usd
0.013030309333333335
output_tokens_per_task
10779.536
latency_seconds_per_task
73.24675466666666
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
36
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

OpenRouter Benchmarks

87.082%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Amazon Bedrock
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
20
sample size
2970
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
15
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Amazon Bedrock
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:49:45.152Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2970
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8708191333333333
accuracy_std_dev
0.005011630059061523
cost_per_task_usd
0.004255378451178452
output_tokens_per_task
7745.265656565656
latency_seconds_per_task
39.48623198653199
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
32
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

OpenRouter Benchmarks

86.717%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Azure
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
27
sample size
1980
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Azure
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T09:34:38.477Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1980
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8671716000000002
accuracy_std_dev
0.006408346388890055
cost_per_task_usd
0.0041782994949494945
output_tokens_per_task
7724.74494949495
latency_seconds_per_task
35.97794848484848
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
43
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

OpenRouter Benchmarks

86.742%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · auto-routing
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
45
sample size
792
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
4
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:27:45.642Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
792
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8674240000000001
accuracy_std_dev
0.0048169925783625705
cost_per_task_usd
0.004064253787878788
output_tokens_per_task
7893.444444444444
latency_seconds_per_task
38.62437373737374
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
132
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

OpenRouter Benchmarks

72.833%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · auto-routing
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
53
sample size
200
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
4
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:31:32.731Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
200
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.72833325
accuracy_std_dev
0.041197373838480274
cost_per_task_usd
0.011627464999999998
output_tokens_per_task
10669.235
latency_seconds_per_task
86.46336500000001
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
128
benchmark_evaluation
true
openrouter_permaslug
anthropic/claude-haiku-5.5-20261007
source model id
anthropic/claude-haiku-5.5-20261007
Original result ↗

Published prices

81–100 / 159

anthropic/claude-haiku-5.5 ↗

$0.55

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
output_tokens_per_million
Currency
USD
Billing conditions
region
scope
specific
geo_region
jp
provider_region
ap-northeast-1
endpoint
bedrock | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
max_exclusive
100001
min_inclusive
0
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$2.75

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
output_tokens_per_million
Currency
USD
Billing conditions
region
scope
zone
geo_region
us
endpoint
bedrock | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
min_inclusive
100001
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.55

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
output_tokens_per_million
Currency
USD
Billing conditions
region
scope
zone
geo_region
us
endpoint
bedrock | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
max_exclusive
100001
min_inclusive
0
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.01

Inference provider
vercelAIGatewayCatalog
Platform
bedrock
Unit
request
Currency
USD
Billing conditions
charge
web_search
endpoint
bedrock | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Web search
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.05

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
endpoint
claudeaws | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
min_inclusive
100001
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.01

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
endpoint
claudeaws | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
max_exclusive
100001
min_inclusive
0
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.63

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
cache_write_tokens_per_million
Currency
USD
Billing conditions
endpoint
claudeaws | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
min_inclusive
100001
Billing label
Cache write / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.13

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
cache_write_tokens_per_million
Currency
USD
Billing conditions
endpoint
claudeaws | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
max_exclusive
100001
min_inclusive
0
Billing label
Cache write / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.50

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
claudeaws | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
min_inclusive
100001
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.10

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
claudeaws | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
max_exclusive
100001
min_inclusive
0
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$2.50

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
output_tokens_per_million
Currency
USD
Billing conditions
endpoint
claudeaws | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
min_inclusive
100001
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.50

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
output_tokens_per_million
Currency
USD
Billing conditions
endpoint
claudeaws | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
context_tokens
max_exclusive
100001
min_inclusive
0
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.01

Inference provider
vercelAIGatewayCatalog
Platform
claudeaws
Unit
request
Currency
USD
Billing conditions
charge
web_search
endpoint
claudeaws | anthropic/claude-haiku-5.5
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Web search
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.05

Inference provider
vercelAIGatewayCatalog
Platform
Vercel AI Gateway
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
model_type
language
quote_scope
gateway_catalog
billing_scope
Vercel public catalog; endpoint prices may differ
context_tokens
min_inclusive
100001
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.01

Inference provider
vercelAIGatewayCatalog
Platform
Vercel AI Gateway
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
model_type
language
quote_scope
gateway_catalog
billing_scope
Vercel public catalog; endpoint prices may differ
context_tokens
max_exclusive
100001
min_inclusive
0
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.06

Inference provider
vercelAIGatewayCatalog
Platform
Vercel AI Gateway
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
region
au
model_type
language
quote_scope
gateway_catalog
billing_scope
Vercel public catalog; endpoint prices may differ
context_tokens
min_inclusive
100001
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.01

Inference provider
vercelAIGatewayCatalog
Platform
Vercel AI Gateway
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
region
au
model_type
language
quote_scope
gateway_catalog
billing_scope
Vercel public catalog; endpoint prices may differ
context_tokens
max_exclusive
100001
min_inclusive
0
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.06

Inference provider
vercelAIGatewayCatalog
Platform
Vercel AI Gateway
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
region
eu
model_type
language
quote_scope
gateway_catalog
billing_scope
Vercel public catalog; endpoint prices may differ
context_tokens
min_inclusive
100001
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.01

Inference provider
vercelAIGatewayCatalog
Platform
Vercel AI Gateway
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
region
eu
model_type
language
quote_scope
gateway_catalog
billing_scope
Vercel public catalog; endpoint prices may differ
context_tokens
max_exclusive
100001
min_inclusive
0
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

anthropic/claude-haiku-5.5 ↗

$0.06

Inference provider
vercelAIGatewayCatalog
Platform
Vercel AI Gateway
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
region
jp
model_type
language
quote_scope
gateway_catalog
billing_scope
Vercel public catalog; endpoint prices may differ
context_tokens
min_inclusive
100001
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

Provider specifications & access

1–4 / 4

Claude Haiku 5.5 ↗

source model id
anthropic/claude-haiku-5.5
hosting
third-party
platforms
  • Anthropic
context length
1000000
max output tokens
128000
architecture
input_modalities
  • text
output_modalities
  • text
capabilities
  • reasoning

Claude Haiku 5.5 ↗

source model id
anthropic/claude-haiku-5.5
hosting
vercel-gateway
platforms
  • Vercel AI Gateway
context length
1000000
max output tokens
128000
architecture
input_modalities
  • text
  • image
  • pdf
output_modalities
  • text
capabilities
  • promptCaching
  • reasoning
  • structuredOutputs
  • toolCalling
  • visionInput
  • webSearch
regions
  • au
  • eu
  • jp
  • us

Claude Haiku 5.5 · azureml-anthropic v1 ↗

source model id
azureml-anthropic/claude-haiku-5-5@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
context length
1000000
max output tokens
128000
architecture
input_modalities
  • text
  • image
  • code
output_modalities
  • text
capabilities
  • reasoning
  • toolCalling

Claude Haiku 5.5 · azureml-anthropic v2 ↗

source model id
azureml-anthropic/claude-haiku-5-5@2
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
context length
1000000
max output tokens
128000
architecture
input_modalities
  • text
  • image
  • code
output_modalities
  • text
capabilities
  • reasoning
  • toolCalling

Safety assessments

1–1 / 1

Foundation Model Transparency Index 2025 ↗

46 / 100

source id
stanfordFMTI
canonical model id
anthropic/claude-haiku-5.5-20261007
taxonomy
transparency
hazard category
transparency
hazard name
Organization transparency
policy area
Anthropic
assessment type
organization-level disclosure
metric
disclosure_score
result label
46 / 100
sample size
100
updated at
2026-11-09T18:08:05.227Z

Source specification observations

1–16 / 16

azureFoundry ↗

field
context_length
value
1000000
normalized value
1000000
scope
source-model
source id
azureFoundry
confidence
1

azureFoundry ↗

field
context_length
value
1000000
normalized value
1000000
scope
source-model
source id
azureFoundry
confidence
1

cloudflareWorkersAI ↗

field
context_length
value
1000000
normalized value
1000000
scope
source-model
source id
cloudflareWorkersAI
confidence
1

openRouter ↗

field
context_length
value
1000000
normalized value
1000000
scope
source-model
source id
openRouter
confidence
1

openRouter ↗

field
context_length
value
1000000
normalized value
1000000
scope
source-model
source id
openRouter
confidence
1

openRouter ↗

field
context_length
value
1000000
normalized value
1000000
scope
source-model
source id
openRouter
confidence
1

vercelAIGatewayCatalog ↗

field
context_length
value
1000000
normalized value
1000000
scope
source-model
source id
vercelAIGatewayCatalog
confidence
1

azureFoundry ↗

field
max_output_tokens
value
128000
normalized value
128000
scope
source-model
source id
azureFoundry
confidence
1

azureFoundry ↗

field
max_output_tokens
value
128000
normalized value
128000
scope
source-model
source id
azureFoundry
confidence
1

cloudflareWorkersAI ↗

field
max_output_tokens
value
128000
normalized value
128000
scope
source-model
source id
cloudflareWorkersAI
confidence
1

openRouter ↗

field
max_output_tokens
value
128000
normalized value
128000
scope
source-model
source id
openRouter
confidence
1

openRouter ↗

field
max_output_tokens
value
128000
normalized value
128000
scope
source-model
source id
openRouter
confidence
1

openRouter ↗

field
max_output_tokens
value
128000
normalized value
128000
scope
source-model
source id
openRouter
confidence
1

vercelAIGatewayCatalog ↗

field
max_output_tokens
value
128000
normalized value
128000
scope
source-model
source id
vercelAIGatewayCatalog
confidence
1

azureFoundry ↗

field
knowledge_cutoff
reported field
training_data_cutoff
value
June 2026
normalized value
2026-06
precision
month
scope
source-model
source id
azureFoundry
confidence
1

azureFoundry ↗

field
knowledge_cutoff
reported field
training_data_cutoff
value
June 2026
normalized value
2026-06
precision
month
scope
source-model
source id
azureFoundry
confidence
1

Original sources

1–16 / 16

Anthropic · Model reference ↗

Microsoft Foundry · Model page ↗

Microsoft Foundry · Model page ↗

OpenRouter · Model page ↗

OpenRouter · Model page ↗

Vercel AI Gateway · Model page ↗

Microsoft Foundry · Model documentation ↗

GitHub · Source snapshot ↗

Cloudflare AI · Catalog ↗

Epoch AI · Catalog ↗

GitHub · Catalog ↗

Microsoft Foundry · Catalog ↗

OpenRouter · Catalog ↗

OpenRouter · Catalog ↗

Vercel AI Gateway · Catalog ↗

Vercel AI Gateway · Catalog ↗

Source links

1–9 / 9

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace