τ-bench

Source-qualified evaluations, report statistics and safety observations.

Claude-3.7-Sonnet

MetricValueUnit
τ-bench · airline · pass^10.642percent
τ-bench · airline · pass^20.589percent
τ-bench · airline · pass^30.5539999999999999percent
τ-bench · airline · pass^40.521percent
τ-bench · retail · pass^10.721percent
τ-bench · retail · pass^20.6679999999999999percent
τ-bench · retail · pass^30.632percent
τ-bench · retail · pass^40.597percent
τ-bench · telecom · pass^10.49percent
τ-bench · telecom · pass^20.43700000000000006percent
τ-bench · telecom · pass^30.401percent
τ-bench · telecom · pass^40.37200000000000005percent

Claude Opus 4.1

MetricValueUnit
τ-bench · airline · pass^10.56percent
τ-bench · retail · pass^10.8240000000000001percent

Claude Opus 4

MetricValueUnit
τ-bench · airline · pass^10.596percent
τ-bench · retail · pass^10.8140000000000001percent

Claude-Sonnet-4.5

MetricValueUnit
τ-bench · airline · pass^10.7percent
τ-bench · retail · pass^10.862percent
τ-bench · telecom · pass^10.98percent

Claude Sonnet 4

MetricValueUnit
τ-bench · airline · pass^10.6percent
τ-bench · retail · pass^10.805percent

DeepSeek-V3.2

MetricValueUnit
τ-bench · airline · pass^10.638percent
τ-bench · retail · pass^10.8109999999999999percent
τ-bench · telecom · pass^10.9620000000000001percent

Gemini 3.0 Pro

MetricValueUnit
τ-bench · airline · pass^10.73percent
τ-bench · retail · pass^10.853percent
τ-bench · telecom · pass^10.98percent

GPT-4.1-mini

MetricValueUnit
τ-bench · airline · pass^10.48700000000000004percent
τ-bench · airline · pass^20.389percent
τ-bench · airline · pass^30.321percent
τ-bench · airline · pass^40.278percent
τ-bench · retail · pass^10.614percent
τ-bench · retail · pass^20.498percent
τ-bench · retail · pass^30.423percent
τ-bench · retail · pass^40.369percent
τ-bench · telecom · pass^10.489percent
τ-bench · telecom · pass^20.387percent
τ-bench · telecom · pass^30.312percent
τ-bench · telecom · pass^40.261percent

GPT-4.1

MetricValueUnit
τ-bench · airline · pass^10.56percent
τ-bench · airline · pass^20.478percent
τ-bench · airline · pass^30.424percent
τ-bench · airline · pass^40.381percent
τ-bench · retail · pass^10.74percent
τ-bench · retail · pass^20.642percent
τ-bench · retail · pass^30.581percent
τ-bench · retail · pass^40.523percent
τ-bench · telecom · pass^10.34percent
τ-bench · telecom · pass^20.276percent
τ-bench · telecom · pass^30.23800000000000002percent
τ-bench · telecom · pass^40.204percent

GPT-5

MetricValueUnit
τ-bench · airline · pass^10.625percent
τ-bench · airline · pass^20.5533333333333332percent
τ-bench · airline · pass^30.51percent
τ-bench · airline · pass^40.48percent
τ-bench · retail · pass^10.8157894736842105percent
τ-bench · retail · pass^20.7178362573099416percent
τ-bench · retail · pass^30.6469298245614036percent
τ-bench · retail · pass^40.5877192982456141percent
τ-bench · telecom · pass^10.9583333333333335percent
τ-bench · telecom · pass^20.9195906432748537percent
τ-bench · telecom · pass^30.8837719298245613percent
τ-bench · telecom · pass^40.8508771929824561percent

Kimi-k2

MetricValueUnit
τ-bench · airline · pass^10.565percent
τ-bench · retail · pass^10.706percent
τ-bench · telecom · pass^10.6579999999999999percent

o3

MetricValueUnit
τ-bench · airline · pass^10.52percent
τ-bench · retail · pass^10.7390000000000001percent

o4-mini

MetricValueUnit
τ-bench · airline · pass^10.521percent
τ-bench · airline · pass^20.442percent
τ-bench · airline · pass^30.389percent
τ-bench · airline · pass^40.34700000000000003percent
τ-bench · retail · pass^10.6829999999999999percent
τ-bench · retail · pass^20.5870000000000001percent
τ-bench · retail · pass^30.521percent
τ-bench · retail · pass^40.46799999999999997percent
τ-bench · telecom · pass^10.502percent
τ-bench · telecom · pass^20.42100000000000004percent
τ-bench · telecom · pass^30.368percent
τ-bench · telecom · pass^40.324percent

Qwen3-Max

MetricValueUnit
τ-bench · airline · pass^10.595percent
τ-bench · airline · pass^20.55percent
τ-bench · airline · pass^40.46percent
τ-bench · retail · pass^10.722percent
τ-bench · retail · pass^20.675percent
τ-bench · retail · pass^40.57percent
τ-bench · telecom · pass^10.8420000000000001percent
τ-bench · telecom · pass^20.7759999999999999percent
τ-bench · telecom · pass^40.614percent

Qwen3-Max-Thinking

MetricValueUnit
τ-bench · airline · pass^10.69percent
τ-bench · airline · pass^20.6533333333333333percent
τ-bench · airline · pass^30.635percent
τ-bench · airline · pass^40.62percent
τ-bench · retail · pass^10.7938596491228069percent
τ-bench · retail · pass^20.6944444444444445percent
τ-bench · retail · pass^30.6271929824561403percent
τ-bench · retail · pass^40.5789473684210527percent
τ-bench · telecom · pass^10.9824561403508771percent
τ-bench · telecom · pass^20.9663742690058479percent
τ-bench · telecom · pass^30.9517543859649122percent
τ-bench · telecom · pass^40.9385964912280701percent

Nemotron-Orchestrator-8B

MetricValueUnit
τ-bench · airline · pass^10.56percent
τ-bench · retail · pass^10.8420000000000001percent
τ-bench · telecom · pass^10.8859999999999999percent

Claude Fable 5

MetricValueUnit
τ-bench · banking_knowledge · pass^10.3969072164948454percent
τ-bench · banking_knowledge · pass^20.32817869415807566percent
τ-bench · banking_knowledge · pass^30.3015463917525773percent
τ-bench · banking_knowledge · pass^40.28865979381443296percent

Claude Opus 4.5

MetricValueUnit
τ-bench · airline · pass^10.84percent
τ-bench · airline · pass^20.7767000000000001percent
τ-bench · airline · pass^30.735percent
τ-bench · airline · pass^40.7percent
τ-bench · banking_knowledge · pass^10.24739999999999998percent
τ-bench · banking_knowledge · pass^20.1718percent
τ-bench · banking_knowledge · pass^30.1366percent
τ-bench · banking_knowledge · pass^40.1134percent
τ-bench · retail · pass^10.7961percent
τ-bench · retail · pass^20.674percent
τ-bench · retail · pass^30.5877percent
τ-bench · retail · pass^40.5175percent
τ-bench · telecom · pass^10.9231999999999999percent
τ-bench · telecom · pass^20.8611percent
τ-bench · telecom · pass^30.8136percent
τ-bench · telecom · pass^40.7807percent

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace