Terminal-Bench
Source-qualified evaluations, report statistics and safety observations.
openai/inkling-small-1102-rc17
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.1303 | percent |
| Terminal-Bench · avg_trial_duration_sec | 7272.7 | seconds |
| Terminal-Bench · pass_at_2 | 0.2 | percent |
| Terminal-Bench · pass_at_3 | 0.2439 | percent |
| Terminal-Bench · pass_at_4 | 0.2758 | percent |
| Terminal-Bench · pass_at_5 | 0.303 | percent |
| Terminal-Bench · total_cost_usd | 963.65 | USD |
openrouter/tencent/hy4-preview
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.19390000000000002 | percent |
| Terminal-Bench · avg_trial_duration_sec | 6967 | seconds |
| Terminal-Bench · pass_at_2 | 0.2773 | percent |
| Terminal-Bench · pass_at_3 | 0.3227 | percent |
| Terminal-Bench · pass_at_4 | 0.3545 | percent |
| Terminal-Bench · pass_at_5 | 0.3788 | percent |
| Terminal-Bench · total_cost_usd | 658.09 | USD |
openrouter/reflection/beam-1.0-501b-a23b
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0636 | percent |
| Terminal-Bench · avg_trial_duration_sec | 178.6 | seconds |
| Terminal-Bench · pass_at_2 | 0.0879 | percent |
| Terminal-Bench · pass_at_3 | 0.0985 | percent |
| Terminal-Bench · pass_at_4 | 0.103 | percent |
| Terminal-Bench · pass_at_5 | 0.1061 | percent |
| Terminal-Bench · total_cost_usd | 220.96 | USD |
deepseek/deepseek-v4.1-flash-1017
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.3848 | percent |
| Terminal-Bench · avg_trial_duration_sec | 148.4 | seconds |
| Terminal-Bench · pass_at_2 | 0.5061 | percent |
| Terminal-Bench · pass_at_3 | 0.5727 | percent |
| Terminal-Bench · pass_at_4 | 0.6182 | percent |
| Terminal-Bench · pass_at_5 | 0.6515 | percent |
| Terminal-Bench · total_cost_usd | 184.89 | USD |
openrouter/deepseek/deepseek-v4-pro-0813
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0667 | percent |
| Terminal-Bench · avg_trial_duration_sec | 138.2 | seconds |
| Terminal-Bench · pass_at_2 | 0.1212 | percent |
| Terminal-Bench · pass_at_3 | 0.1652 | percent |
| Terminal-Bench · pass_at_4 | 0.2 | percent |
| Terminal-Bench · pass_at_5 | 0.2273 | percent |
| Terminal-Bench · total_cost_usd | 727.02 | USD |
z-ai/glm-5.3-flash
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.3697 | percent |
| Terminal-Bench · avg_trial_duration_sec | 120.4 | seconds |
| Terminal-Bench · pass_at_2 | 0.4833 | percent |
| Terminal-Bench · pass_at_3 | 0.5424 | percent |
| Terminal-Bench · pass_at_4 | 0.5879 | percent |
| Terminal-Bench · pass_at_5 | 0.6212 | percent |
| Terminal-Bench · total_cost_usd | 374.72 | USD |
openai/gpt-5.4-mini
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0182 | percent |
| Terminal-Bench · avg_trial_duration_sec | 141.5 | seconds |
| Terminal-Bench · pass_at_2 | 0.0318 | percent |
| Terminal-Bench · pass_at_3 | 0.0424 | percent |
| Terminal-Bench · pass_at_4 | 0.0515 | percent |
| Terminal-Bench · pass_at_5 | 0.0606 | percent |
| Terminal-Bench · total_cost_usd | 671.57 | USD |
openai/gpt-6.1-terra
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.5333 | percent |
| Terminal-Bench · avg_trial_duration_sec | 123.1 | seconds |
| Terminal-Bench · pass_at_2 | 0.6152 | percent |
| Terminal-Bench · pass_at_3 | 0.6485 | percent |
| Terminal-Bench · pass_at_4 | 0.6636 | percent |
| Terminal-Bench · pass_at_5 | 0.6667 | percent |
| Terminal-Bench · total_cost_usd | 874.44 | USD |
anthropic/claude-haiku-5-5
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.4 | percent |
| Terminal-Bench · avg_trial_duration_sec | 140.5 | seconds |
| Terminal-Bench · pass_at_2 | 0.5121 | percent |
| Terminal-Bench · pass_at_3 | 0.5636 | percent |
| Terminal-Bench · pass_at_4 | 0.5909 | percent |
| Terminal-Bench · pass_at_5 | 0.6061 | percent |
| Terminal-Bench · total_cost_usd | 952.47 | USD |
openrouter/tencent/hy4
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.3788 | percent |
| Terminal-Bench · avg_trial_duration_sec | 126.2 | seconds |
| Terminal-Bench · pass_at_2 | 0.4848 | percent |
| Terminal-Bench · pass_at_3 | 0.5394 | percent |
| Terminal-Bench · pass_at_4 | 0.5727 | percent |
| Terminal-Bench · pass_at_5 | 0.5909 | percent |
| Terminal-Bench · total_cost_usd | 347.6 | USD |
openrouter/thinkingmachines/inkling
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0 | percent |
| Terminal-Bench · avg_trial_duration_sec | 174.1 | seconds |
| Terminal-Bench · pass_at_2 | 0 | percent |
| Terminal-Bench · pass_at_3 | 0 | percent |
| Terminal-Bench · pass_at_4 | 0 | percent |
| Terminal-Bench · pass_at_5 | 0 | percent |
| Terminal-Bench · total_cost_usd | 1011.52 | USD |
openrouter/thinkingmachines/inkling-small
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0091 | percent |
| Terminal-Bench · avg_trial_duration_sec | 141.3 | seconds |
| Terminal-Bench · pass_at_2 | 0.0167 | percent |
| Terminal-Bench · pass_at_3 | 0.0227 | percent |
| Terminal-Bench · pass_at_4 | 0.0273 | percent |
| Terminal-Bench · pass_at_5 | 0.0303 | percent |
| Terminal-Bench · total_cost_usd | 49.55 | USD |
moonshotai/kimi-k3.1-pro
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.597 | percent |
| Terminal-Bench · avg_trial_duration_sec | 123.2 | seconds |
| Terminal-Bench · pass_at_2 | 0.747 | percent |
| Terminal-Bench · pass_at_3 | 0.8076 | percent |
| Terminal-Bench · pass_at_4 | 0.8364 | percent |
| Terminal-Bench · pass_at_5 | 0.8485 | percent |
| Terminal-Bench · total_cost_usd | 1086.66 | USD |
openrouter/moonshotai/kimi-k3
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.2303 | percent |
| Terminal-Bench · avg_trial_duration_sec | 141.9 | seconds |
| Terminal-Bench · pass_at_2 | 0.35 | percent |
| Terminal-Bench · pass_at_3 | 0.4273 | percent |
| Terminal-Bench · pass_at_4 | 0.4848 | percent |
| Terminal-Bench · pass_at_5 | 0.5303 | percent |
| Terminal-Bench · total_cost_usd | 1919.83 | USD |
openrouter/xiaomi/mimo-v2.6-flash
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.1606 | percent |
| Terminal-Bench · avg_trial_duration_sec | 160.8 | seconds |
| Terminal-Bench · pass_at_2 | 0.2576 | percent |
| Terminal-Bench · pass_at_3 | 0.3182 | percent |
| Terminal-Bench · pass_at_4 | 0.3576 | percent |
| Terminal-Bench · pass_at_5 | 0.3788 | percent |
| Terminal-Bench · total_cost_usd | 194.6 | USD |
openrouter/xiaomi/mimo-v2.6-pro
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.2697 | percent |
| Terminal-Bench · avg_trial_duration_sec | 168.2 | seconds |
| Terminal-Bench · pass_at_2 | 0.3818 | percent |
| Terminal-Bench · pass_at_3 | 0.447 | percent |
| Terminal-Bench · pass_at_4 | 0.4939 | percent |
| Terminal-Bench · pass_at_5 | 0.5303 | percent |
| Terminal-Bench · total_cost_usd | 282.32 | USD |
openrouter/minimax/minimax-m3
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0758 | percent |
| Terminal-Bench · avg_trial_duration_sec | 204 | seconds |
| Terminal-Bench · pass_at_2 | 0.1409 | percent |
| Terminal-Bench · pass_at_3 | 0.1955 | percent |
| Terminal-Bench · pass_at_4 | 0.2394 | percent |
| Terminal-Bench · pass_at_5 | 0.2727 | percent |
| Terminal-Bench · total_cost_usd | 1149.33 | USD |
openrouter/mistralai/mistral-large-4-0
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.2758 | percent |
| Terminal-Bench · avg_trial_duration_sec | 142.7 | seconds |
| Terminal-Bench · pass_at_2 | 0.4045 | percent |
| Terminal-Bench · pass_at_3 | 0.4848 | percent |
| Terminal-Bench · pass_at_4 | 0.5394 | percent |
| Terminal-Bench · pass_at_5 | 0.5758 | percent |
| Terminal-Bench · total_cost_usd | 718.4 | USD |