LiveCodeBench
Source-qualified evaluations, report statistics and safety observations.
Kimi-k1.6-IOI
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.8014772727272726 | percent |
Kimi-k1.6-IOI-high
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.8602272727272726 | percent |
Llama-3_1-Nemotron-Nano-8B-v1
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.6443181818181819 | percent |
Llama-3_1-Nemotron-Ultra-253B-v1
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.7772727272727273 | percent |
MetaStone-L1-7B
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.5409090909090909 | percent |
Mistral-Large
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.3729545454545455 | percent |
O1-2024-12-17 (High)
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.8318181818181819 | percent |
O1-2024-12-17 (Low)
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.759090909090909 | percent |
O1-2024-12-17 (Med)
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.7829545454545455 | percent |
O1-Mini-2024-09-12
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.6840909090909091 | percent |
O1-Preview-2024-09-12
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.5556818181818182 | percent |
O3-Mini-2025-01-31 (High)
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.8045454545454546 | percent |
O3-Mini-2025-01-31 (Low)
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.7454545454545455 | percent |
O3-Mini-2025-01-31 (Med)
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.7920454545454545 | percent |
O4-Mini (High)
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.8556818181818181 | percent |
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.5988636363636364 | percent |
QwQ-Max-Preview
| Metric | Value | Unit |
|---|
| LiveCodeBench v5 · derived cohort mean | 0.8 | percent |