LiveCodeBench
Source-qualified evaluations, report statistics and safety observations.
AzeroGPT-64b
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.2203409090909091 | percent |
Claude-3.5-Sonnet-20240620
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.48034090909090904 | percent |
Claude-3.5-Sonnet-20241022
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.5146590909090909 | percent |
Claude-3-Haiku
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.22181818181818183 | percent |
Codestral-Latest
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.3532954545454545 | percent |
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.7329545454545454 | percent |
DeepSeek-R1-Lite-Preview
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.6307954545454545 | percent |
DeepSeek-R1-Preview
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.7788636363636364 | percent |
DeepSeek-V3
derived mean pass@1 · No aggregate inferred percent
Gemini-2.5-Pro
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.8465909090909091 | percent |
Gemini-Flash-1.5-002
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.36125 | percent |
Gemini-Flash-2.0-Exp
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.41590909090909095 | percent |
Gemini-Pro-1.5-002
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.42090909090909095 | percent |
GPT-4O-2024-05-13
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.4343181818181818 | percent |
GPT-4O-2024-08-06
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.4018181818181818 | percent |
GPT-4O-mini-2024-07-18
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.3718181818181818 | percent |
GPT-4-Turbo-2024-04-09
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.39090909090909093 | percent |
Grok-3-Mini (High)
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.8136363636363636 | percent |