Z.ai: GLM 4.7
GLM-4.7 is Z.ai’s latest flagship model, featuring upgrades in two key areas: enhanced programming capabilities and more stable multi-step reasoning/execution. It demonstrates significant improvement…
z-ai/glm-4.7-20251222
GLM-4.7 is Z.ai’s latest flagship model, featuring upgrades in two key areas: enhanced programming capabilities and more stable multi-step reasoning/execution. It demonstrates significant improvements in executing complex agent tasks while delivering more natural conversational experiences and superior front-end aesthetics.
Specifications & capabilities
- Input
- text
- Output
- text
- Context window · tokens
- 204800
- Knowledge cutoff
- 2025-04
- Published model ID
- GLM-4.7
- Capabilities
- promptCaching, reasoning, structuredOutputs, toolCalling
Description source ↗Evaluations
1–20 / 26
83.8%
- source id
- openCompass
- leaderboard name
- LiveCodeBenchV6
- dimension
- general
- metric
- LiveCodeBenchV6
- rank
- 3
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- LiveCodeBenchV6
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- GLM-4.7
Original result ↗95.4%
- source id
- openCompass
- leaderboard name
- AIME2025
- dimension
- general
- metric
- AIME2025
- rank
- 4
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- AIME2025
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- GLM-4.7
Original result ↗86.9%
- source id
- openCompass
- leaderboard name
- GPQA-Diamond
- dimension
- general
- metric
- GPQA-Diamond
- rank
- 4
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- GPQA-Diamond
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- GLM-4.7
Original result ↗83.694%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Z.AI
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 4
- sample size
- 1386
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Z.AI
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T16:08:56.903Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1386
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8369407142857143
- accuracy_std_dev
- 0.020626950634783515
- cost_per_task_usd
- 0.10078755411255412
- output_tokens_per_task
- 46038.86075036075
- latency_seconds_per_task
- 1217.5330151515152
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 8
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗70.095%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Z.AI
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 6
- sample size
- 350
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Z.AI
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T20:44:35.166Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 350
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7009522857142857
- accuracy_std_dev
- 0.04471111302801668
- cost_per_task_usd
- 0.06413979428571429
- output_tokens_per_task
- 10011.842857142858
- latency_seconds_per_task
- 345.33378285714286
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 8
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗56.086%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Mancer 2
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 6
- sample size
- 1142
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Mancer 2
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T16:02:18.348Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1142
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.5608581453590192
- accuracy_std_dev
- 0.2662797085149478
- cost_per_task_usd
- 0.09908757443082312
- output_tokens_per_task
- 42074.91681260946
- latency_seconds_per_task
- 1949.5623021015763
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 6
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗77.6%
- source id
- openCompass
- leaderboard name
- Average
- dimension
- general
- metric
- Average
- rank
- 7
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- Average
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- GLM-4.7
Original result ↗25.4%
- source id
- openCompass
- leaderboard name
- HLE
- dimension
- general
- metric
- HLE
- rank
- 7
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- HLE
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- GLM-4.7
Original result ↗84%
- source id
- openCompass
- leaderboard name
- MMLU-Pro
- dimension
- general
- metric
- MMLU-Pro
- rank
- 12
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- MMLU-Pro
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- GLM-4.7
Original result ↗84.259%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · AtlasCloud
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 14
- sample size
- 396
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- AtlasCloud
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-04T00:07:08.217Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 396
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8425925
- accuracy_std_dev
- 0.005892500000000023
- cost_per_task_usd
- 0.08320908333333334
- output_tokens_per_task
- 44847.378787878784
- latency_seconds_per_task
- 696.7031691919192
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 30
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗75.2%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Google
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 14
- sample size
- 250
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Google
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-17T19:14:08.120Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 250
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.752
- accuracy_std_dev
- 0.018570921118781412
- cost_per_task_usd
- 0.12117575600000001
- output_tokens_per_task
- 10483.008
- latency_seconds_per_task
- 124.707216
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 36
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗72.333%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · StreamLake
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 14
- sample size
- 100
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- StreamLake
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-08-30T00:01:09.490Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 100
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7233335
- accuracy_std_dev
- 0.04333349999999997
- cost_per_task_usd
- 0.11253119
- output_tokens_per_task
- 10275.49
- latency_seconds_per_task
- 335.49034
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 25
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗90.2%
- source id
- openCompass
- leaderboard name
- IFEval
- dimension
- general
- metric
- IFEval
- rank
- 15
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- IFEval
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- GLM-4.7
Original result ↗73.333%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · AtlasCloud
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 15
- sample size
- 100
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- AtlasCloud
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-04T00:00:34.896Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 100
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7333330000000001
- accuracy_std_dev
- 0.020000000000000018
- cost_per_task_usd
- 0.055889249999999994
- output_tokens_per_task
- 9141.8
- latency_seconds_per_task
- 201.94525
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 29
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗82.071%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · StreamLake
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 19
- sample size
- 396
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- StreamLake
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-08-30T00:37:59.094Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 396
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8207070000000001
- accuracy_std_dev
- 0.007575999999999972
- cost_per_task_usd
- 0.07039754797979798
- output_tokens_per_task
- 40130.17171717172
- latency_seconds_per_task
- 879.5930631313132
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 30
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗83.333%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Google
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 22
- sample size
- 990
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Google
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-17T16:06:16.399Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 990
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8333332
- accuracy_std_dev
- 0.00946378413532349
- cost_per_task_usd
- 0.10132244444444444
- output_tokens_per_task
- 46075.3101010101
- latency_seconds_per_task
- 359.966904040404
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 40
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗71.524%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Venice
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 24
- sample size
- 350
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Venice
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T19:41:08.087Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 350
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7152381428571428
- accuracy_std_dev
- 0.022172375614898876
- cost_per_task_usd
- 0.04021302285714286
- output_tokens_per_task
- 9178.234285714285
- latency_seconds_per_task
- 623.0208714285714
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 38
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗84.007%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Novita
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 25
- sample size
- 1386
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Novita
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T16:33:40.856Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1386
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8400672857142857
- accuracy_std_dev
- 0.010723216824312801
- cost_per_task_usd
- 0.09338691269841272
- output_tokens_per_task
- 47107.84848484849
- latency_seconds_per_task
- 1164.2649783549784
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 59
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗80.991%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · DeepInfra
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 29
- sample size
- 612
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- DeepInfra
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T16:05:14.633Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 612
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8099128055555557
- accuracy_std_dev
- 0.02251018815558855
- cost_per_task_usd
- 0.06893759803921569
- output_tokens_per_task
- 39280.651960784315
- latency_seconds_per_task
- 1733.7049052287582
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 71
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗71.833%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · DeepInfra
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 33
- sample size
- 200
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 4
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- DeepInfra
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T21:01:10.349Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 200
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.71833325
- accuracy_std_dev
- 0.03378866920563018
- cost_per_task_usd
- 0.03334888
- output_tokens_per_task
- 8985.465
- latency_seconds_per_task
- 414.66168
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 63
- benchmark_evaluation
- true
- openrouter_permaslug
- z-ai/glm-4.7-20251222
- source model id
- z-ai/glm-4.7-20251222
Original result ↗Published prices
1–18 / 18
$0.08
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.40
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$1.75
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.42
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
$2.20
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
$0.60
- Inference provider
- vercelAIGatewayCatalog
- Platform
- bedrock
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- bedrock | zai/glm-4.7
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$2.20
- Inference provider
- vercelAIGatewayCatalog
- Platform
- bedrock
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- bedrock | zai/glm-4.7
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.08
- Inference provider
- vercelAIGatewayCatalog
- Platform
- deepinfra
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- deepinfra | zai/glm-4.7
- model_type
- language
- quantization
- fp4
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.40
- Inference provider
- vercelAIGatewayCatalog
- Platform
- deepinfra
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- deepinfra | zai/glm-4.7
- model_type
- language
- quantization
- fp4
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$1.75
- Inference provider
- vercelAIGatewayCatalog
- Platform
- deepinfra
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- deepinfra | zai/glm-4.7
- model_type
- language
- quantization
- fp4
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.11
- Inference provider
- vercelAIGatewayCatalog
- Platform
- novita
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- novita | zai/glm-4.7
- model_type
- language
- quantization
- fp8
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.60
- Inference provider
- vercelAIGatewayCatalog
- Platform
- novita
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- novita | zai/glm-4.7
- model_type
- language
- quantization
- fp8
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$2.20
- Inference provider
- vercelAIGatewayCatalog
- Platform
- novita
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- novita | zai/glm-4.7
- model_type
- language
- quantization
- fp8
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.60
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- varies_by_provider
- true
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$2.20
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- varies_by_provider
- true
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.11
- Inference provider
- vercelAIGatewayCatalog
- Platform
- zai
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- zai | zai/glm-4.7
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.60
- Inference provider
- vercelAIGatewayCatalog
- Platform
- zai
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- zai | zai/glm-4.7
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$2.20
- Inference provider
- vercelAIGatewayCatalog
- Platform
- zai
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- zai | zai/glm-4.7
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–9 / 9
- source model id
- azure-huggingface/zai-org--glm-4.7@1
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 202752
- source model id
- azure-huggingface/zai-org--glm-4.7@2
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 202752
- source model id
- azure-huggingface/zai-org--glm-4.7@3
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 202752
- source model id
- azure-huggingface/zai-org--glm-4.7@4
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 202752
- source model id
- azure-huggingface/zai-org--glm-4.7@5
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 202752
- source model id
- azure-huggingface/zai-org--glm-4.7@6
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 202752
- source model id
- azure-huggingface/zai-org--glm-4.7@7
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 202752
- source model id
- zai/glm-4.7
- hosting
- vercel-gateway
- context length
- 200000
- max output tokens
- 120000
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- source model id
- zai-org/GLM-4.7
- hosting
- siliconflow-hosted
- max output tokens
- 204800
Source specification observations
1–14 / 14
- field
- context_length
- value
- 202752
- normalized value
- 202752
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 202752
- normalized value
- 202752
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 202752
- normalized value
- 202752
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 202752
- normalized value
- 202752
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 202752
- normalized value
- 202752
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 202752
- normalized value
- 202752
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 202752
- normalized value
- 202752
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 204800
- normalized value
- 204800
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 205K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-04
- normalized value
- 2025-04
- precision
- month
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 131072
- normalized value
- 131072
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 205K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- max_output_tokens
- value
- 120000
- normalized value
- 120000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
Explore more models