Terminal-Bench
Source-qualified evaluations, report statistics and safety observations.
openai/gpt-6-sol
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.4939 | percent |
| Terminal-Bench · avg_trial_duration_sec | 2662.2 | seconds |
| Terminal-Bench · pass_at_2 | 0.5833 | percent |
| Terminal-Bench · pass_at_3 | 0.6242 | percent |
| Terminal-Bench · pass_at_4 | 0.6485 | percent |
| Terminal-Bench · pass_at_5 | 0.6667 | percent |
| Terminal-Bench · total_cost_usd | 1451.32 | USD |
qwen/qwen3.8-max-0902
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.2697 | percent |
| Terminal-Bench · avg_trial_duration_sec | 9953 | seconds |
| Terminal-Bench · pass_at_2 | 0.3924 | percent |
| Terminal-Bench · pass_at_3 | 0.4591 | percent |
| Terminal-Bench · pass_at_4 | 0.5 | percent |
| Terminal-Bench · pass_at_5 | 0.5303 | percent |
| Terminal-Bench · total_cost_usd | 4635.67 | USD |
z-ai/glm-5.3-flash
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.3576 | percent |
| Terminal-Bench · avg_trial_duration_sec | 6209.8 | seconds |
| Terminal-Bench · pass_at_2 | 0.4652 | percent |
| Terminal-Bench · pass_at_3 | 0.5136 | percent |
| Terminal-Bench · pass_at_4 | 0.5424 | percent |
| Terminal-Bench · pass_at_5 | 0.5606 | percent |
| Terminal-Bench · total_cost_usd | 118.73752448 | USD |
meta/muse-wonder-1.0
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.4697 | percent |
| Terminal-Bench · avg_trial_duration_sec | 2827.5 | seconds |
| Terminal-Bench · pass_at_2 | 0.5924 | percent |
| Terminal-Bench · pass_at_3 | 0.6545 | percent |
| Terminal-Bench · pass_at_4 | 0.6909 | percent |
| Terminal-Bench · pass_at_5 | 0.7121 | percent |
| Terminal-Bench · total_cost_usd | 2452.31 | USD |
openai/gpt-5.4-mini
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0182 | percent |
| Terminal-Bench · avg_trial_duration_sec | 1848.5 | seconds |
| Terminal-Bench · pass_at_2 | 0.0318 | percent |
| Terminal-Bench · pass_at_3 | 0.0424 | percent |
| Terminal-Bench · pass_at_4 | 0.0515 | percent |
| Terminal-Bench · pass_at_5 | 0.0606 | percent |
| Terminal-Bench · total_cost_usd | 671.57 | USD |
anthropic/claude-haiku-5-5
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.4 | percent |
| Terminal-Bench · avg_trial_duration_sec | 4059 | seconds |
| Terminal-Bench · pass_at_2 | 0.5167 | percent |
| Terminal-Bench · pass_at_3 | 0.5727 | percent |
| Terminal-Bench · pass_at_4 | 0.603 | percent |
| Terminal-Bench · pass_at_5 | 0.6212 | percent |
| Terminal-Bench · total_cost_usd | 952.47 | USD |
openrouter/thinkingmachines/inkling
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0 | percent |
| Terminal-Bench · avg_trial_duration_sec | 1402.6 | seconds |
| Terminal-Bench · pass_at_2 | 0 | percent |
| Terminal-Bench · pass_at_3 | 0 | percent |
| Terminal-Bench · pass_at_4 | 0 | percent |
| Terminal-Bench · pass_at_5 | 0 | percent |
| Terminal-Bench · total_cost_usd | 1011.52 | USD |
openrouter/thinkingmachines/inkling-small
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0091 | percent |
| Terminal-Bench · avg_trial_duration_sec | 475.1 | seconds |
| Terminal-Bench · pass_at_2 | 0.0167 | percent |
| Terminal-Bench · pass_at_3 | 0.0227 | percent |
| Terminal-Bench · pass_at_4 | 0.0273 | percent |
| Terminal-Bench · pass_at_5 | 0.0303 | percent |
| Terminal-Bench · total_cost_usd | 49.55 | USD |
openrouter/reflection/beam-1.0-501b-a23b
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0636 | percent |
| Terminal-Bench · avg_trial_duration_sec | 1440.1 | seconds |
| Terminal-Bench · pass_at_2 | 0.0879 | percent |
| Terminal-Bench · pass_at_3 | 0.0985 | percent |
| Terminal-Bench · pass_at_4 | 0.103 | percent |
| Terminal-Bench · pass_at_5 | 0.1061 | percent |
| Terminal-Bench · total_cost_usd | 233 | USD |
openrouter/nvidia/nemotron-3.5-super-vl
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.003 | percent |
| Terminal-Bench · avg_trial_duration_sec | 1780.6 | seconds |
| Terminal-Bench · pass_at_2 | 0.0061 | percent |
| Terminal-Bench · pass_at_3 | 0.0091 | percent |
| Terminal-Bench · pass_at_4 | 0.0121 | percent |
| Terminal-Bench · pass_at_5 | 0.0152 | percent |
| Terminal-Bench · total_cost_usd | 167.29 | USD |
anthropic/claude-fable-5-2
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.6576000000000001 | percent |
| Terminal-Bench · avg_trial_duration_sec | 7333 | seconds |
| Terminal-Bench · pass_at_2 | 0.7273 | percent |
| Terminal-Bench · pass_at_3 | 0.7545 | percent |
| Terminal-Bench · pass_at_4 | 0.7727 | percent |
| Terminal-Bench · pass_at_5 | 0.7879 | percent |
| Terminal-Bench · total_cost_usd | 16492.76 | USD |
meta/muse-ember-1.0
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.3061 | percent |
| Terminal-Bench · avg_trial_duration_sec | 2288.4 | seconds |
| Terminal-Bench · pass_at_2 | 0.4303 | percent |
| Terminal-Bench · pass_at_3 | 0.503 | percent |
| Terminal-Bench · pass_at_4 | 0.5545 | percent |
| Terminal-Bench · pass_at_5 | 0.5909 | percent |
| Terminal-Bench · total_cost_usd | 157.18 | USD |
openrouter/deepseek/deepseek-v4-pro-0813
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0667 | percent |
| Terminal-Bench · avg_trial_duration_sec | 3140 | seconds |
| Terminal-Bench · pass_at_2 | 0.1212 | percent |
| Terminal-Bench · pass_at_3 | 0.1652 | percent |
| Terminal-Bench · pass_at_4 | 0.2 | percent |
| Terminal-Bench · pass_at_5 | 0.2273 | percent |
| Terminal-Bench · total_cost_usd | 727.02 | USD |
openrouter/minimax/minimax-m3
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0727 | percent |
| Terminal-Bench · avg_trial_duration_sec | 3507 | seconds |
| Terminal-Bench · pass_at_2 | 0.1348 | percent |
| Terminal-Bench · pass_at_3 | 0.1864 | percent |
| Terminal-Bench · pass_at_4 | 0.2273 | percent |
| Terminal-Bench · pass_at_5 | 0.2576 | percent |
| Terminal-Bench · total_cost_usd | 1149.33 | USD |
openrouter/moonshotai/kimi-k3
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.2242 | percent |
| Terminal-Bench · avg_trial_duration_sec | 3351 | seconds |
| Terminal-Bench · pass_at_2 | 0.3485 | percent |
| Terminal-Bench · pass_at_3 | 0.4273 | percent |
| Terminal-Bench · pass_at_4 | 0.4848 | percent |
| Terminal-Bench · pass_at_5 | 0.5303 | percent |
| Terminal-Bench · total_cost_usd | 1919.83 | USD |
openrouter/tencent/hy4
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.3909 | percent |
| Terminal-Bench · avg_trial_duration_sec | 4881.6 | seconds |
| Terminal-Bench · pass_at_2 | 0.4955 | percent |
| Terminal-Bench · pass_at_3 | 0.55 | percent |
| Terminal-Bench · pass_at_4 | 0.5848 | percent |
| Terminal-Bench · pass_at_5 | 0.6061 | percent |
| Terminal-Bench · total_cost_usd | 347.6 | USD |
openrouter/xiaomi/mimo-v2.6-flash
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.1606 | percent |
| Terminal-Bench · avg_trial_duration_sec | 5471.6 | seconds |
| Terminal-Bench · pass_at_2 | 0.2576 | percent |
| Terminal-Bench · pass_at_3 | 0.3182 | percent |
| Terminal-Bench · pass_at_4 | 0.3576 | percent |
| Terminal-Bench · pass_at_5 | 0.3788 | percent |
| Terminal-Bench · total_cost_usd | 194.6 | USD |
openrouter/xiaomi/mimo-v2.6-pro
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.2697 | percent |
| Terminal-Bench · avg_trial_duration_sec | 5265.3 | seconds |
| Terminal-Bench · pass_at_2 | 0.3818 | percent |
| Terminal-Bench · pass_at_3 | 0.447 | percent |
| Terminal-Bench · pass_at_4 | 0.4939 | percent |
| Terminal-Bench · pass_at_5 | 0.5303 | percent |
| Terminal-Bench · total_cost_usd | 282.32 | USD |