Thinking Machines: Inkling 2 Small (free)

Inkling 2 Small is a multimodal mixture-of-experts reasoning model from Thinking Machines Lab, with 17B active parameters out of 594B total. It is the first model in the second Inkling generation. It…

thinkingmachines/inkling-2-small-20261107

Inkling 2 Small is a multimodal mixture-of-experts reasoning model from Thinking Machines Lab, with 17B active parameters out of 594B total. It is the first model in the second Inkling generation. It was trained with reinforcement learning on long-horizon coding tasks across many agent harnesses, and is suited for agentic coding, autonomous research, agentic knowledge work and retrieval-augmented generation, with English and general multilingual support. It accepts text and image input (no audio), supports tool calling and variable reasoning effort, and has a 1M-token context window.

Specifications & capabilities

Input
text, image
Output
text
Context window · tokens
1048576
Capabilities
multimodal, reasoning, toolCalling, visionInput
Description source ↗

Evaluations

1–13 / 13

OpenRouter Benchmarks

91.077%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · PrimeIntellect
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
1
sample size
198
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
PrimeIntellect
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T03:01:57.471Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
198
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.910774
cost_per_task_usd
0.01510519191919192
output_tokens_per_task
30029.237373737375
latency_seconds_per_task
254.02976767676768
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
22
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

91.077%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Cloudflare
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
2
sample size
198
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Cloudflare
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T00:01:53.427Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
198
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.910774
cost_per_task_usd
0.030777131313131315
output_tokens_per_task
30575.68686868687
latency_seconds_per_task
94.52724747474748
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
18
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

91.414%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Nebius
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
3
sample size
198
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T00:01:53.399Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
198
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.9141410000000001
cost_per_task_usd
0.030628545454545457
output_tokens_per_task
30417.116161616163
latency_seconds_per_task
320.02058585858583
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

91.414%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Together
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
3
sample size
198
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Together
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T00:01:54.553Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
198
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.9141410000000001
cost_per_task_usd
0.03261640909090909
output_tokens_per_task
32384.540404040403
latency_seconds_per_task
98.34765151515153
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

90.909%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Modal
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
4
sample size
198
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Modal
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T06:01:37.481Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
198
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.9090909999999999
cost_per_task_usd
0.02941411111111111
output_tokens_per_task
29214.535353535353
latency_seconds_per_task
293.24638383838385
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
11
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

90.236%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · DeepInfra
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
9
sample size
198
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
DeepInfra
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T12:01:34.311Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
198
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.902357
cost_per_task_usd
0.027153818181818182
output_tokens_per_task
26931.89898989899
latency_seconds_per_task
429.4187121212121
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
71
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

63.333%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Modal
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
10
sample size
50
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Modal
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T06:54:36.403Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
50
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.633333
cost_per_task_usd
0.042546600000000004
output_tokens_per_task
26828.8
latency_seconds_per_task
200.10172
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
11
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

63.333%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · PrimeIntellect
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
12
sample size
50
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
PrimeIntellect
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T03:43:05.942Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
50
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.633333
cost_per_task_usd
0.01754236
output_tokens_per_task
8415.52
latency_seconds_per_task
112.67148
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
21
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

62.667%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Cloudflare
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
15
sample size
50
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Cloudflare
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T00:15:45.631Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
50
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.626667
cost_per_task_usd
0.0507923
output_tokens_per_task
11296.32
latency_seconds_per_task
58.77358
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
15
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

65.333%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Nebius
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
17
sample size
50
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T00:42:42.289Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
50
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.653333
cost_per_task_usd
0.032076099999999996
output_tokens_per_task
8232.88
latency_seconds_per_task
129.95052
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
23
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

64%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Together
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
20
sample size
50
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Together
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T00:16:41.107Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
50
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.64
cost_per_task_usd
0.026583100000000002
output_tokens_per_task
8435.96
latency_seconds_per_task
49.49466
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
22
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

OpenRouter Benchmarks

70.667%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · DeepInfra
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
35
sample size
50
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
DeepInfra
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-13T12:51:19.323Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
50
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.706667
cost_per_task_usd
0.023244639999999997
output_tokens_per_task
8529.56
latency_seconds_per_task
174.55852
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
63
benchmark_evaluation
true
openrouter_permaslug
thinkingmachines/inkling-2-small-20261107
source model id
thinkingmachines/inkling-2-small-20261107
Original result ↗

Published prices

1–2 / 2

Source specification observations

1–2 / 2

openRouter ↗

field
context_length
value
1048576
normalized value
1048576
scope
source-model
source id
openRouter
confidence
1

openRouter ↗

field
max_output_tokens
value
131072
normalized value
131072
scope
source-model
source id
openRouter
confidence
1

Original sources

1–3 / 3

OpenRouter · Model page ↗

OpenRouter · Catalog ↗

OpenRouter · Catalog ↗

Source links

1–3 / 3

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace