τ-bench

Source-qualified evaluations, report statistics and safety observations.

Claude Opus 4.6

MetricValueUnit
τ-bench · banking_knowledge · pass^10.2732percent
τ-bench · banking_knowledge · pass^20.1873percent
τ-bench · banking_knowledge · pass^30.1418percent
τ-bench · banking_knowledge · pass^40.1134percent

Claude Opus 4.7

MetricValueUnit
τ-bench · banking_knowledge · pass^10.4020618556701031percent
τ-bench · banking_knowledge · pass^20.3109965635738832percent
τ-bench · banking_knowledge · pass^30.2706185567010309percent
τ-bench · banking_knowledge · pass^40.24742268041237114percent

Claude Opus 4.8

MetricValueUnit
τ-bench · banking_knowledge · pass^10.3969072164948454percent
τ-bench · banking_knowledge · pass^20.31271477663230246percent
τ-bench · banking_knowledge · pass^30.26288659793814434percent
τ-bench · banking_knowledge · pass^40.2268041237113402percent

Claude Opus 5

MetricValueUnit
τ-bench · banking_knowledge · pass^10.48711340206185566percent
τ-bench · banking_knowledge · pass^20.3917525773195877percent
τ-bench · banking_knowledge · pass^30.34536082474226804percent
τ-bench · banking_knowledge · pass^40.31958762886597936percent

Claude Sonnet 4.5

MetricValueUnit
τ-bench · airline · pass^10.72percent
τ-bench · airline · pass^20.62percent
τ-bench · airline · pass^30.545percent
τ-bench · airline · pass^40.48percent
τ-bench · banking_knowledge · pass^10.2526percent
τ-bench · banking_knowledge · pass^20.15810000000000002percent
τ-bench · banking_knowledge · pass^30.1263percent
τ-bench · banking_knowledge · pass^40.10310000000000001percent
τ-bench · retail · pass^10.7237percent
τ-bench · retail · pass^20.5746percent
τ-bench · retail · pass^30.47369999999999995percent
τ-bench · retail · pass^40.3947percent
τ-bench · telecom · pass^10.8487percent
τ-bench · telecom · pass^20.7544percent
τ-bench · telecom · pass^30.6886percent
τ-bench · telecom · pass^40.6404000000000001percent

PrincipledMachines Cortex 0.1

MetricValueUnit
τ-bench · banking_knowledge · pass^10.7319587628865979percent

Distyl ButtonAgent

MetricValueUnit
τ-bench · banking_knowledge · pass^10.3119percent
τ-bench · banking_knowledge · pass^20.2147percent
τ-bench · banking_knowledge · pass^30.16190000000000002percent
τ-bench · banking_knowledge · pass^40.134percent

Gemini 2.5 Pro

MetricValueUnit
τ-bench · banking_knowledge · pass^10.1366percent
τ-bench · banking_knowledge · pass^20.0601percent
τ-bench · banking_knowledge · pass^30.0258percent
τ-bench · banking_knowledge · pass^40.0103percent

Gemini 3.1 Pro Preview

MetricValueUnit
τ-bench · banking_knowledge · pass^10.26030000000000003percent
τ-bench · banking_knowledge · pass^20.1512percent
τ-bench · banking_knowledge · pass^30.1108percent
τ-bench · banking_knowledge · pass^40.0928percent

Gemini 3 Flash

MetricValueUnit
τ-bench · airline · pass^10.825percent
τ-bench · airline · pass^20.7633percent
τ-bench · airline · pass^30.72percent
τ-bench · airline · pass^40.68percent
τ-bench · banking_knowledge · pass^10.2732percent
τ-bench · banking_knowledge · pass^20.15460000000000002percent
τ-bench · banking_knowledge · pass^30.10310000000000001percent
τ-bench · banking_knowledge · pass^40.0722percent
τ-bench · retail · pass^10.7675percent
τ-bench · retail · pass^20.6594percent
τ-bench · retail · pass^30.5789percent
τ-bench · retail · pass^40.5175percent
τ-bench · telecom · pass^10.9123percent
τ-bench · telecom · pass^20.8348percent
τ-bench · telecom · pass^30.7654000000000001percent
τ-bench · telecom · pass^40.7018000000000001percent

Gemini 3 Pro

MetricValueUnit
τ-bench · airline · pass^10.805percent
τ-bench · airline · pass^20.7467percent
τ-bench · airline · pass^30.7percent
τ-bench · airline · pass^40.66percent
τ-bench · banking_knowledge · pass^10.1804percent
τ-bench · banking_knowledge · pass^20.0928percent
τ-bench · banking_knowledge · pass^30.0567percent
τ-bench · banking_knowledge · pass^40.0412percent
τ-bench · retail · pass^10.7587999999999999percent
τ-bench · retail · pass^20.6345000000000001percent
τ-bench · retail · pass^30.5439percent
τ-bench · retail · pass^40.47369999999999995percent
τ-bench · telecom · pass^10.9101percent
τ-bench · telecom · pass^20.845percent
τ-bench · telecom · pass^30.7917000000000001percent
τ-bench · telecom · pass^40.7456percent

GLM-5.2

MetricValueUnit
τ-bench · banking_knowledge · pass^10.3711340206185567percent
τ-bench · banking_knowledge · pass^20.25257731958762886percent
τ-bench · banking_knowledge · pass^30.18298969072164947percent
τ-bench · banking_knowledge · pass^40.13402061855670103percent

GLM-5

MetricValueUnit
τ-bench · airline · pass^10.825percent
τ-bench · airline · pass^20.76percent
τ-bench · airline · pass^30.72percent
τ-bench · airline · pass^40.7percent
τ-bench · banking_knowledge · pass^10.09789999999999999percent
τ-bench · banking_knowledge · pass^20.0722percent
τ-bench · banking_knowledge · pass^30.0567percent
τ-bench · banking_knowledge · pass^40.030899999999999997percent
τ-bench · retail · pass^10.7368000000000001percent
τ-bench · retail · pass^20.6038percent
τ-bench · retail · pass^30.511percent
τ-bench · retail · pass^40.4386percent
τ-bench · telecom · pass^10.8684000000000001percent
τ-bench · telecom · pass^20.7631999999999999percent
τ-bench · telecom · pass^30.682percent
τ-bench · telecom · pass^40.6228percent

GPT-5.2

MetricValueUnit
τ-bench · airline · pass^10.525percent
τ-bench · airline · pass^20.3566666666666667percent
τ-bench · airline · pass^30.27percent
τ-bench · airline · pass^40.22percent
τ-bench · banking_knowledge · pass^10.1263percent
τ-bench · banking_knowledge · pass^20.0722percent
τ-bench · banking_knowledge · pass^30.051500000000000004percent
τ-bench · banking_knowledge · pass^40.0412percent
τ-bench · retail · pass^10.75percent
τ-bench · retail · pass^20.6213450292397662percent
τ-bench · retail · pass^30.5307017543859649percent
τ-bench · retail · pass^40.45614035087719296percent
τ-bench · telecom · pass^10.5723684210526315percent
τ-bench · telecom · pass^20.4283625730994152percent
τ-bench · telecom · pass^30.35526315789473684percent
τ-bench · telecom · pass^40.30701754385964913percent

GPT-5.2

MetricValueUnit
τ-bench · airline · pass^10.83percent
τ-bench · airline · pass^20.7833percent
τ-bench · airline · pass^30.75percent
τ-bench · airline · pass^40.72percent
τ-bench · banking_knowledge · pass^10.3222percent
τ-bench · banking_knowledge · pass^20.23879999999999998percent
τ-bench · banking_knowledge · pass^30.20620000000000002percent
τ-bench · banking_knowledge · pass^40.1856percent
τ-bench · retail · pass^10.8158percent
τ-bench · retail · pass^20.6959000000000001percent
τ-bench · retail · pass^30.5987percent
τ-bench · retail · pass^40.5175percent
τ-bench · telecom · pass^10.8969percent
τ-bench · telecom · pass^20.8245999999999999percent
τ-bench · telecom · pass^30.7675percent
τ-bench · telecom · pass^40.7193percent

GPT-5.4

MetricValueUnit
τ-bench · banking_knowledge · pass^10.3943percent
τ-bench · banking_knowledge · pass^20.2955percent
τ-bench · banking_knowledge · pass^30.24480000000000002percent
τ-bench · banking_knowledge · pass^40.2165percent

GPT-5.5

MetricValueUnit
τ-bench · banking_knowledge · pass^10.44587628865979384percent
τ-bench · banking_knowledge · pass^20.37628865979381454percent
τ-bench · banking_knowledge · pass^30.3324742268041237percent
τ-bench · banking_knowledge · pass^40.29896907216494845percent

GPT-5.6-sol

MetricValueUnit
τ-bench · banking_knowledge · pass^10.46907216494845355percent
τ-bench · banking_knowledge · pass^20.3728522336769759percent
τ-bench · banking_knowledge · pass^30.31958762886597936percent
τ-bench · banking_knowledge · pass^40.27835051546391754percent

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace