Terminal-Bench
Source-qualified evaluations, report statistics and safety observations.
openrouter/nvidia/nemotron-3.5-super-vl
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.003 | percent |
| Terminal-Bench · avg_trial_duration_sec | 135.8 | seconds |
| Terminal-Bench · pass_at_2 | 0.0061 | percent |
| Terminal-Bench · pass_at_3 | 0.0091 | percent |
| Terminal-Bench · pass_at_4 | 0.0121 | percent |
| Terminal-Bench · pass_at_5 | 0.0152 | percent |
| Terminal-Bench · total_cost_usd | 167.29 | USD |
gemini/gemini-4-argon
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.5485 | percent |
| Terminal-Bench · avg_trial_duration_sec | 3894.8 | seconds |
| Terminal-Bench · pass_at_2 | 0.6242 | percent |
| Terminal-Bench · pass_at_3 | 0.6485 | percent |
| Terminal-Bench · pass_at_4 | 0.6606 | percent |
| Terminal-Bench · pass_at_5 | 0.6667 | percent |
| Terminal-Bench · total_cost_usd | 1108.98 | USD |
gemini/gemini-4-argon
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.5545 | percent |
| Terminal-Bench · avg_trial_duration_sec | 125.2 | seconds |
| Terminal-Bench · pass_at_2 | 0.6242 | percent |
| Terminal-Bench · pass_at_3 | 0.6439 | percent |
| Terminal-Bench · pass_at_4 | 0.6515 | percent |
| Terminal-Bench · pass_at_5 | 0.6515 | percent |
| Terminal-Bench · total_cost_usd | 1108.98 | USD |
gemini/gemini-4-hydrogen
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.5636 | percent |
| Terminal-Bench · avg_trial_duration_sec | 1926.2 | seconds |
| Terminal-Bench · pass_at_2 | 0.6394 | percent |
| Terminal-Bench · pass_at_3 | 0.6712 | percent |
| Terminal-Bench · pass_at_4 | 0.6879 | percent |
| Terminal-Bench · pass_at_5 | 0.697 | percent |
| Terminal-Bench · total_cost_usd | 784.24 | USD |
meta/muse-wonder-1.0
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.4697 | percent |
| Terminal-Bench · avg_trial_duration_sec | 2827.5 | seconds |
| Terminal-Bench · pass_at_2 | 0.5924 | percent |
| Terminal-Bench · pass_at_3 | 0.6545 | percent |
| Terminal-Bench · pass_at_4 | 0.6909 | percent |
| Terminal-Bench · pass_at_5 | 0.7121 | percent |
| Terminal-Bench · total_cost_usd | 2452.31 | USD |
gemini/gemini-3.1-pro-preview
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.0273 | percent |
| Terminal-Bench · avg_trial_duration_sec | 4583.2 | seconds |
| Terminal-Bench · pass_at_2 | 0.0515 | percent |
| Terminal-Bench · pass_at_3 | 0.0727 | percent |
| Terminal-Bench · pass_at_4 | 0.0909 | percent |
| Terminal-Bench · pass_at_5 | 0.1061 | percent |
| Terminal-Bench · total_cost_usd | 1971.51 | USD |
gemini/gemini-4-carbon
| Metric | Value | Unit |
|---|
| Terminal-Bench · accuracy | 0.6455 | percent |
| Terminal-Bench · avg_trial_duration_sec | 3495.8 | seconds |
| Terminal-Bench · pass_at_2 | 0.7348 | percent |
| Terminal-Bench · pass_at_3 | 0.7682 | percent |
| Terminal-Bench · pass_at_4 | 0.7818 | percent |
| Terminal-Bench · pass_at_5 | 0.7879 | percent |
| Terminal-Bench · total_cost_usd | 1358.25 | USD |