Z.ai: GLM 5.1

GLM-5.1 delivers a major leap in coding capability, with particularly significant gains in handling long-horizon tasks. Unlike previous models built around minute-level interactions, GLM-5.1 can work…

z-ai/glm-5.1-20260406

GLM-5.1 delivers a major leap in coding capability, with particularly significant gains in handling long-horizon tasks. Unlike previous models built around minute-level interactions, GLM-5.1 can work independently and continuously on a single task for more than 8 hours, autonomously planning, executing, and improving itself throughout the process, ultimately delivering complete, engineering-grade results.

Specifications & capabilities

Input
text
Output
text
Context window · tokens
204800
Published model ID
GLM-5.1
Capabilities
promptCaching, reasoning, structuredOutputs, toolCalling
Description source ↗

Evaluations

21–37 / 37

OpenRouter Benchmarks

72.833%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Phala
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
15
sample size
200
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
4
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Phala
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T22:21:14.715Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
200
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.72833325
accuracy_std_dev
0.03411588502717615
cost_per_task_usd
0.17819147999999999
output_tokens_per_task
9652.53
latency_seconds_per_task
442.72501500000004
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
23
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

70.25%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Nebius
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
15
sample size
400
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
8
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T22:14:51.015Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
400
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7025
accuracy_std_dev
0.030173028796426787
cost_per_task_usd
0.2793706775
output_tokens_per_task
9455.475
latency_seconds_per_task
496.47328749999997
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
23
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

82.758%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Nebius
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
16
sample size
1187
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
6
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:22:14.129Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1187
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8275762754844144
accuracy_std_dev
0.06091378546606307
cost_per_task_usd
0.20493896293176075
output_tokens_per_task
46988.62679022746
latency_seconds_per_task
1744.7425619208088
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

72.381%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · SiliconFlow
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
17
sample size
350
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
SiliconFlow
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T19:57:59.015Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
350
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7238094285714286
accuracy_std_dev
0.03062471250410662
cost_per_task_usd
0.1589893657142857
output_tokens_per_task
9576.722857142857
latency_seconds_per_task
227.08273428571428
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
28
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

74.145%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · DigitalOcean
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
19
sample size
156
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
DigitalOcean
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-08-22T17:09:42.218Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
156
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.741453
cost_per_task_usd
0.09924664102564103
output_tokens_per_task
29914.98717948718
latency_seconds_per_task
1908.5117756410257
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
23
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

82.443%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · SiliconFlow
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
22
sample size
1386
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
SiliconFlow
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:50:02.353Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1386
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8244347142857142
accuracy_std_dev
0.01399044581863215
cost_per_task_usd
0.1977735064935065
output_tokens_per_task
52647.930735930735
latency_seconds_per_task
958.8975014430015
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
34
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

75.346%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · GMICloud
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
23
sample size
1782
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
9
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
GMICloud
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-10T07:04:16.019Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1782
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7534605555555556
accuracy_std_dev
0.03816866389275402
cost_per_task_usd
0.350709655443322
output_tokens_per_task
84600.8125701459
latency_seconds_per_task
957.2535684624017
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
26
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

78.187%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · AtlasCloud
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
27
sample size
1386
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
AtlasCloud
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:04:02.366Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1386
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7818664285714286
accuracy_std_dev
0.016143829133821997
cost_per_task_usd
0.3387973888888889
output_tokens_per_task
85385.09307359307
latency_seconds_per_task
1030.179549062049
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
30
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

74%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · DeepInfra
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
27
sample size
50
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
DeepInfra
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-08-22T16:12:20.461Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
50
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.74
cost_per_task_usd
0.0873214
output_tokens_per_task
8891.3
latency_seconds_per_task
347.03732
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
63
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

81.073%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Novita
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
34
sample size
1386
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Novita
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:12:28.234Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1386
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8107262857142856
accuracy_std_dev
0.02121111496843299
cost_per_task_usd
0.29609035353535357
output_tokens_per_task
67260.45959595959
latency_seconds_per_task
1026.6817662337662
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
59
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

71.333%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Novita
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
34
sample size
350
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Novita
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T20:31:35.900Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
350
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7133332857142858
accuracy_std_dev
0.026427450856552726
cost_per_task_usd
0.13545089142857142
output_tokens_per_task
10590.037142857143
latency_seconds_per_task
267.52531714285715
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
52
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

77.333%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · DeepInfra
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
38
sample size
175
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
DeepInfra
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-08-22T17:30:52.646Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
175
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.773333
cost_per_task_usd
0.13656869142857142
output_tokens_per_task
38871.68571428571
latency_seconds_per_task
1521.7931542857143
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
71
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

71.714%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · auto-routing
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
62
sample size
350
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T20:09:58.111Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
350
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7171427142857143
accuracy_std_dev
0.03802358976708421
cost_per_task_usd
0.10077471142857143
output_tokens_per_task
10145.26857142857
latency_seconds_per_task
217.1396
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
128
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

OpenRouter Benchmarks

80.988%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · auto-routing
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
77
sample size
1386
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:12:28.230Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1386
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8098844285714284
accuracy_std_dev
0.01753250978575957
cost_per_task_usd
0.23733018614718615
output_tokens_per_task
63860.35858585859
latency_seconds_per_task
1080.6483585858584
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
132
benchmark_evaluation
true
openrouter_permaslug
z-ai/glm-5.1-20260406
source model id
z-ai/glm-5.1-20260406
Original result ↗

Published prices

1–15 / 15

z-ai/glm-5.1 ↗

$0.26

Inference provider
openRouter
Platform
OpenRouter
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions

z-ai/glm-5.1 ↗

$1.40

Inference provider
openRouter
Platform
OpenRouter
Unit
input_tokens_per_million
Currency
USD
Billing conditions

z-ai/glm-5.1 ↗

$4.40

Inference provider
openRouter
Platform
OpenRouter
Unit
output_tokens_per_million
Currency
USD
Billing conditions

zai-org/GLM-5.1 ↗

$0.60

Inference provider
siliconFlow
Platform
SiliconFlow
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
Billing label
Cached input tokens (per 1M)

zai-org/GLM-5.1 ↗

$1.19

Inference provider
siliconFlow
Platform
SiliconFlow
Unit
input_tokens_per_million
Currency
USD
Billing conditions
Billing label
Input tokens (per 1M)

zai-org/GLM-5.1 ↗

$3.74

Inference provider
siliconFlow
Platform
SiliconFlow
Unit
output_tokens_per_million
Currency
USD
Billing conditions
Billing label
Output tokens (per 1M)

zai/glm-5.1 ↗

$0.26

Inference provider
vercelAIGatewayCatalog
Platform
novita
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
endpoint
novita | zai/glm-5.1
model_type
language
quantization
fp8
billing_scope
via Vercel AI Gateway
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

zai/glm-5.1 ↗

$1.38

Inference provider
vercelAIGatewayCatalog
Platform
novita
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
novita | zai/glm-5.1
model_type
language
quantization
fp8
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

zai/glm-5.1 ↗

$4.40

Inference provider
vercelAIGatewayCatalog
Platform
novita
Unit
output_tokens_per_million
Currency
USD
Billing conditions
endpoint
novita | zai/glm-5.1
model_type
language
quantization
fp8
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

zai/glm-5.1 ↗

$0.26

Inference provider
vercelAIGatewayCatalog
Platform
Vercel AI Gateway
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
model_type
language
quote_scope
gateway_catalog
billing_scope
Vercel public catalog; endpoint prices may differ
varies_by_provider
true
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

zai/glm-5.1 ↗

$1.40

Inference provider
vercelAIGatewayCatalog
Platform
Vercel AI Gateway
Unit
input_tokens_per_million
Currency
USD
Billing conditions
model_type
language
quote_scope
gateway_catalog
billing_scope
Vercel public catalog; endpoint prices may differ
varies_by_provider
true
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

zai/glm-5.1 ↗

$4.40

Inference provider
vercelAIGatewayCatalog
Platform
Vercel AI Gateway
Unit
output_tokens_per_million
Currency
USD
Billing conditions
model_type
language
quote_scope
gateway_catalog
billing_scope
Vercel public catalog; endpoint prices may differ
varies_by_provider
true
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

zai/glm-5.1 ↗

$0.26

Inference provider
vercelAIGatewayCatalog
Platform
zai
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
endpoint
zai | zai/glm-5.1
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

zai/glm-5.1 ↗

$1.40

Inference provider
vercelAIGatewayCatalog
Platform
zai
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
zai | zai/glm-5.1
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

zai/glm-5.1 ↗

$4.40

Inference provider
vercelAIGatewayCatalog
Platform
zai
Unit
output_tokens_per_million
Currency
USD
Billing conditions
endpoint
zai | zai/glm-5.1
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

Provider specifications & access

1–2 / 2

GLM 5.1 ↗

source model id
zai/glm-5.1
hosting
vercel-gateway
platforms
  • Vercel AI Gateway
context length
202800
max output tokens
64000
architecture
input_modalities
  • text
output_modalities
  • text
capabilities
  • promptCaching
  • reasoning
  • structuredOutputs
  • toolCalling

GLM-5.1 ↗

source model id
zai-org/GLM-5.1
hosting
siliconflow-hosted
platforms
  • SiliconFlow
max output tokens
131072
architecture
input_modalities
  • text
output_modalities
  • text
capabilities
  • promptCaching
  • reasoning
  • toolCalling

Source specification observations

1–6 / 6

openRouter ↗

field
context_length
value
204800
normalized value
204800
scope
source-model
source id
openRouter
confidence
1

siliconFlow ↗

field
context_length
value
205K
scope
source-model
source id
siliconFlow
confidence
1

vercelAIGatewayCatalog ↗

field
context_length
value
202800
normalized value
202800
scope
source-model
source id
vercelAIGatewayCatalog
confidence
1

openRouter ↗

field
max_output_tokens
value
182476
normalized value
182476
scope
source-model
source id
openRouter
confidence
1

siliconFlow ↗

field
max_output_tokens
value
131K
scope
source-model
source id
siliconFlow
confidence
1

vercelAIGatewayCatalog ↗

field
max_output_tokens
value
64000
normalized value
64000
scope
source-model
source id
vercelAIGatewayCatalog
confidence
1

Original sources

1–15 / 15

SiliconFlow · Model page ↗

z.ai · Model reference ↗

Hugging Face · Model repository ↗

OpenRouter · Model page ↗

SiliconFlow · Playground ↗

Vercel AI Gateway · Model page ↗

SiliconFlow · API documentation ↗

arXiv · Paper · 2602.15763 ↗

Epoch AI · Catalog ↗

Hugging Face · Catalog ↗

Hugging Face · Catalog ↗

OpenRouter · Catalog ↗

OpenRouter · Catalog ↗

SiliconFlow · Catalog ↗

Vercel AI Gateway · Catalog ↗

Source links

1–8 / 8

Hugging Face model card

1–1 / 1

zai-org/GLM-5.1 ↗

downloads
878904
likes
1850
pipeline tag
text-generation
library name
transformers
License
mit

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace