Terminal-Bench
Source-qualified evaluations, report statistics and safety observations.
anthropic/claude-fable-5
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.44549999999999995 | percent |
| Terminal-Bench · avg_trial_duration_sec | 4202.8 | seconds |
| Terminal-Bench · pass_at_2 | 0.5727 | percent |
| Terminal-Bench · pass_at_3 | 0.6318 | percent |
| Terminal-Bench · pass_at_4 | 0.6636 | percent |
| Terminal-Bench · pass_at_5 | 0.6818 | percent |
| Terminal-Bench · total_cost_usd | 7265.01 | USD |
anthropic/claude-opus-4-8
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.2364 | percent |
| Terminal-Bench · avg_trial_duration_sec | 5163 | seconds |
| Terminal-Bench · pass_at_2 | 0.3455 | percent |
| Terminal-Bench · pass_at_3 | 0.4045 | percent |
| Terminal-Bench · pass_at_4 | 0.4424 | percent |
| Terminal-Bench · pass_at_5 | 0.4697 | percent |
| Terminal-Bench · total_cost_usd | 6481.26 | USD |
anthropic/claude-opus-5
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.5182 | percent |
| Terminal-Bench · avg_trial_duration_sec | 4792.8 | seconds |
| Terminal-Bench · pass_at_2 | 0.6167 | percent |
| Terminal-Bench · pass_at_3 | 0.6576 | percent |
| Terminal-Bench · pass_at_4 | 0.6818 | percent |
| Terminal-Bench · pass_at_5 | 0.697 | percent |
| Terminal-Bench · total_cost_usd | 5969.11 | USD |
anthropic/claude-sonnet-5
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.1242 | percent |
| Terminal-Bench · avg_trial_duration_sec | 6510.3 | seconds |
| Terminal-Bench · pass_at_2 | 0.2045 | percent |
| Terminal-Bench · pass_at_3 | 0.2621 | percent |
| Terminal-Bench · pass_at_4 | 0.3091 | percent |
| Terminal-Bench · pass_at_5 | 0.3485 | percent |
| Terminal-Bench · total_cost_usd | 9603.86 | USD |
anthropic/glm-5.3
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.4182 | percent |
| Terminal-Bench · avg_trial_duration_sec | 5831 | seconds |
| Terminal-Bench · pass_at_2 | 0.5076 | percent |
| Terminal-Bench · pass_at_3 | 0.547 | percent |
| Terminal-Bench · pass_at_4 | 0.5667 | percent |
| Terminal-Bench · pass_at_5 | 0.5758 | percent |
| Terminal-Bench · total_cost_usd | 2727.63 | USD |
openai/gpt-5.6-luna
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.1727 | percent |
| Terminal-Bench · avg_trial_duration_sec | 4088.3 | seconds |
| Terminal-Bench · pass_at_2 | 0.2424 | percent |
| Terminal-Bench · pass_at_3 | 0.2848 | percent |
| Terminal-Bench · pass_at_4 | 0.3121 | percent |
| Terminal-Bench · pass_at_5 | 0.3333 | percent |
| Terminal-Bench · total_cost_usd | 346.67 | USD |
openai/gpt-5.6-sol
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.37270000000000003 | percent |
| Terminal-Bench · avg_trial_duration_sec | 2388.4 | seconds |
| Terminal-Bench · pass_at_2 | 0.4955 | percent |
| Terminal-Bench · pass_at_3 | 0.55 | percent |
| Terminal-Bench · pass_at_4 | 0.5818 | percent |
| Terminal-Bench · pass_at_5 | 0.6061 | percent |
| Terminal-Bench · total_cost_usd | 2541.7 | USD |
openai/gpt-5.6-terra
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.2152 | percent |
| Terminal-Bench · avg_trial_duration_sec | 2514.6 | seconds |
| Terminal-Bench · pass_at_2 | 0.3061 | percent |
| Terminal-Bench · pass_at_3 | 0.3636 | percent |
| Terminal-Bench · pass_at_4 | 0.4061 | percent |
| Terminal-Bench · pass_at_5 | 0.4394 | percent |
| Terminal-Bench · total_cost_usd | 1733.52 | USD |
xai/grok-4.5
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.1242 | percent |
| Terminal-Bench · avg_trial_duration_sec | 3162.1 | seconds |
| Terminal-Bench · pass_at_2 | 0.1833 | percent |
| Terminal-Bench · pass_at_3 | 0.2227 | percent |
| Terminal-Bench · pass_at_4 | 0.2515 | percent |
| Terminal-Bench · pass_at_5 | 0.2727 | percent |
| Terminal-Bench · total_cost_usd | 2094.11 | USD |
xai/grok-4.6
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.203 | percent |
| Terminal-Bench · avg_trial_duration_sec | 2351.5 | seconds |
| Terminal-Bench · pass_at_2 | 0.2848 | percent |
| Terminal-Bench · pass_at_3 | 0.3303 | percent |
| Terminal-Bench · pass_at_4 | 0.3636 | percent |
| Terminal-Bench · pass_at_5 | 0.3939 | percent |
| Terminal-Bench · total_cost_usd | 3591.58 | USD |
gemini/gemini-3.8-flash
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.1909 | percent |
| Terminal-Bench · avg_trial_duration_sec | 1995.1 | seconds |
| Terminal-Bench · pass_at_2 | 0.2879 | percent |
| Terminal-Bench · pass_at_3 | 0.353 | percent |
| Terminal-Bench · pass_at_4 | 0.4 | percent |
| Terminal-Bench · pass_at_5 | 0.4394 | percent |
| Terminal-Bench · total_cost_usd | 0 | USD |
anthropic/claude-fable-5-1
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.5788 | percent |
| Terminal-Bench · avg_trial_duration_sec | 3893.5 | seconds |
| Terminal-Bench · pass_at_2 | 0.7 | percent |
| Terminal-Bench · pass_at_3 | 0.747 | percent |
| Terminal-Bench · pass_at_4 | 0.7727 | percent |
| Terminal-Bench · pass_at_5 | 0.7879 | percent |
| Terminal-Bench · total_cost_usd | 6243.5 | USD |
gemini/gemini-3.7-flash
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.1121 | percent |
| Terminal-Bench · avg_trial_duration_sec | 1688.9 | seconds |
| Terminal-Bench · pass_at_2 | 0.1636 | percent |
| Terminal-Bench · pass_at_3 | 0.2 | percent |
| Terminal-Bench · pass_at_4 | 0.2303 | percent |
| Terminal-Bench · pass_at_5 | 0.2576 | percent |
| Terminal-Bench · total_cost_usd | 1261.87 | USD |
anthropic/claude-opus-5-5
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.6485 | percent |
| Terminal-Bench · avg_trial_duration_sec | 4790.4 | seconds |
| Terminal-Bench · pass_at_2 | 0.7318 | percent |
| Terminal-Bench · pass_at_3 | 0.7652 | percent |
| Terminal-Bench · pass_at_4 | 0.7788 | percent |
| Terminal-Bench · pass_at_5 | 0.7879 | percent |
| Terminal-Bench · total_cost_usd | 4718.65 | USD |
anthropic/claude-sonnet-5-5
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.6182 | percent |
| Terminal-Bench · avg_trial_duration_sec | 7431.6 | seconds |
| Terminal-Bench · pass_at_2 | 0.6924 | percent |
| Terminal-Bench · pass_at_3 | 0.7364 | percent |
| Terminal-Bench · pass_at_4 | 0.7606 | percent |
| Terminal-Bench · pass_at_5 | 0.7727 | percent |
| Terminal-Bench · total_cost_usd | 7333.64 | USD |
meta/muse-spark-1.3
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.14550000000000002 | percent |
| Terminal-Bench · avg_trial_duration_sec | 4341.7 | seconds |
| Terminal-Bench · pass_at_2 | 0.2197 | percent |
| Terminal-Bench · pass_at_3 | 0.2652 | percent |
| Terminal-Bench · pass_at_4 | 0.3 | percent |
| Terminal-Bench · pass_at_5 | 0.3333 | percent |
| Terminal-Bench · total_cost_usd | 0 | USD |
openai/gpt-6.1-sol
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.5818 | percent |
| Terminal-Bench · avg_trial_duration_sec | 3176.7 | seconds |
| Terminal-Bench · pass_at_2 | 0.6652 | percent |
| Terminal-Bench · pass_at_3 | 0.7015 | percent |
| Terminal-Bench · pass_at_4 | 0.7242 | percent |
| Terminal-Bench · pass_at_5 | 0.7424 | percent |
| Terminal-Bench · total_cost_usd | 634.24 | USD |
openai/gpt-6-luna
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.1636 | percent |
| Terminal-Bench · avg_trial_duration_sec | 2284.1 | seconds |
| Terminal-Bench · pass_at_2 | 0.2273 | percent |
| Terminal-Bench · pass_at_3 | 0.2621 | percent |
| Terminal-Bench · pass_at_4 | 0.2848 | percent |
| Terminal-Bench · pass_at_5 | 0.303 | percent |
| Terminal-Bench · total_cost_usd | 72.14 | USD |