abacusai/Llama-3-Smaug-8B
mmlu_pro · 36.93 source-defined score; unit unspecified
Source-qualified evaluations, report statistics and safety observations.
mmlu_pro · 36.93 source-defined score; unit unspecified
brier · 0.162 source-defined score; unit unspecified
MathArena/hmmt_feb_2026 · 100 source-defined score; unit unspecified
long · 4.99 source-defined score; unit unspecified
old_scans · 85.19 source-defined score; unit unspecified
mmlu_pro · 81.2 source-defined score; unit unspecified
time · 0.8333 source-defined score; unit unspecified
text_formatting · 47 source-defined score; unit unspecified
old_scans · 42.6 source-defined score; unit unspecified
MathArena/aime_2026 · 94.7 source-defined score; unit unspecified
terminal_bench · 52.4 source-defined score; unit unspecified
illumination · 89.66 source-defined score; unit unspecified
avg_cost · 40.08 source-defined score; unit unspecified
mmlu_pro · 37.1 source-defined score; unit unspecified
headers_footers · 78.26 source-defined score; unit unspecified
SWE_Bench_Pro · 58.4 source-defined score; unit unspecified
ar · 56.6 source-defined score; unit unspecified
de · 72.4 source-defined score; unit unspecified
These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.