Open Ko-LLM Leaderboard

Indexed results from Open Ko-LLM Leaderboard. Each ranking retains its own metric and evaluation conditions.

kornat_harmless · acc_norm,none

openKoLLM · Open Ko-LLM Leaderboard · kornat_harmless · acc_norm,none · percent · Higher is better

  1. F24/llama-2-koen-orca-mini-platypus2-math-13bf24100%
  2. MoaData/Myrrh_solar_10.7b_3.0moadata88.803%
  3. tanitan2/SOLAR-10.7B-Instruct-T3Q-ko-solar-dpo-tiestanitan288.043%
  4. tanitan2/SOLAR-10.7B-Instruct-STOCK_SOLAR-tiestanitan288.033%
  5. chihoonlee10/T3Q-ko-solar-dpo-v7.0chihoonlee1086.627%
  6. freewheelin/free-solar-evo-v0.13freewheelin85.08%
  7. freewheelin/free-solar-evo-v0.11freewheelin83.853%
  8. freewheelin/free-solar-evo-v0.1freewheelin83.833%
  9. chihoonlee10/T3Q-ko-solar-dpo-v3.0chihoonlee1081.863%
  10. Edentns/DataVortexS-10.7B-dpo-v1.6edentns80.09%
  11. mlabonne/NeuralMonarch-7Bmlabonne79.697%
  12. chihoonlee10/T3Q-ko-solar-dpo-v1.0chihoonlee1079.68%

ko_gpqa_diamond_zeroshot · acc_norm,none

openKoLLM · Open Ko-LLM Leaderboard · ko_gpqa_diamond_zeroshot · acc_norm,none · percent · Higher is better

  1. BAAI/Infinity-Instruct-7M-Gen-Llama3_1-8Bbaai28.788%
  2. BAAI/Infinity-Instruct-7M-Gen-Llama3_1-8Bbaai28.788%
  3. chujiezheng/Mistral7B-PairRM-SPPO-ExPOchujiezheng28.788%
  4. princeton-nlp/gemma-2-9b-it-SimPOprinceton-nlp28.788%
  5. princeton-nlp/Mistral-7B-Base-SFT-RDPOprinceton-nlp28.788%
  6. v000000/Qwen2.5-14B-Gutenberg-1e-Deltav00000028.788%
  7. Magpie-Align/MagpieLM-8B-Chat-v0.1magpie-align28.283%
  8. anthracite-org/magnum-v3-27b-ktoanthracite-org27.778%
  9. cognitivecomputations/dolphin-2.9.3-mistral-nemo-12bcognitivecomputations27.778%
  10. princeton-nlp/Llama-3-Base-8B-SFT-DPOprinceton-nlp27.778%
  11. princeton-nlp/Llama-3-Instruct-8B-DPO-v0.2princeton-nlp27.778%
  12. v000000/Qwen2.5-14B-Gutenberg-Instruct-Slerpenov00000027.778%

ko_ifeval · prompt_level_strict_acc,none

openKoLLM · Open Ko-LLM Leaderboard · ko_ifeval · prompt_level_strict_acc,none · percent · Higher is better

  1. nbeerbower/gemma2-gutenberg-27Bnbeerbower72.065%
  2. byroneverson/gemma-2-27b-it-abliteratedbyroneverson69.636%
  3. recoilme/recoilme-gemma-2-9B-v0.2recoilme65.992%
  4. v000000/Qwen2.5-14B-Gutenberg-Instruct-Slerpenov00000065.182%
  5. v000000/Qwen2.5-14B-Gutenberg-1e-Deltav00000064.777%
  6. v000000/Qwen2.5-Lumen-14Bv00000064.777%
  7. rombodawg/Rombos-LLM-V2.6-Qwen-14brombodawg64.575%
  8. UCLA-AGI/Gemma-2-9B-It-SPPO-Iter1ucla-agi64.17%
  9. arcee-ai/SuperNova-Mediusarcee-ai63.968%
  10. UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3ucla-agi63.563%
  11. Lambent/qwen2.5-reinstruct-alternate-lumen-14Blambent63.36%
  12. Qwen/Qwen2.5-14B-Instructqwen63.36%

kornat_common · acc_norm,none

openKoLLM · Open Ko-LLM Leaderboard · kornat_common · acc_norm,none · percent · Higher is better

  1. tanliboy/lambda-qwen2.5-14b-dpo-testtanliboy46.255%
  2. v000000/Qwen2.5-Lumen-14Bv00000040.097%
  3. v000000/Qwen2.5-14B-Gutenberg-Instruct-Slerpenov00000039.847%
  4. v000000/Qwen2.5-14B-Gutenberg-1e-Deltav00000039.68%
  5. Lambent/qwen2.5-reinstruct-alternate-lumen-14Blambent39.664%
  6. Qwen/Qwen2.5-14B-Instructqwen39.564%
  7. recoilme/recoilme-gemma-2-9B-v0.4recoilme39.314%
  8. rombodawg/Rombos-LLM-V2.6-Qwen-14brombodawg39.214%
  9. byroneverson/gemma-2-27b-it-abliteratedbyroneverson38.748%
  10. nbeerbower/gemma2-gutenberg-27Bnbeerbower38.049%
  11. arcee-ai/SuperNova-Mediusarcee-ai37.8%
  12. AALF/gemma-2-27b-it-SimPO-37K-100stepsaalf36.634%

ko_eqbench · eqbench,none

openKoLLM · Open Ko-LLM Leaderboard · ko_eqbench · eqbench,none · score · Higher is better

  1. recoilme/Gemma-2-Ataraxy-Gemmasutra-9B-slerprecoilme62.532
  2. UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3ucla-agi62.403
  3. recoilme/recoilme-gemma-2-9B-v0.4recoilme61.433
  4. UCLA-AGI/Gemma-2-9B-It-SPPO-Iter2ucla-agi61.399
  5. UCLA-AGI/Gemma-2-9B-It-SPPO-Iter1ucla-agi61.391
  6. TheDrummer/Gemmasutra-9B-v1thedrummer60.302
  7. Saxo/Linkbricks-Horizon-AI-Korean-Superb-22Bsaxo59.769
  8. Lambent/qwen2.5-reinstruct-alternate-lumen-14Blambent59.258
  9. v000000/Qwen2.5-14B-Gutenberg-1e-Deltav00000059.251
  10. v000000/Qwen2.5-Lumen-14Bv00000059.227
  11. arcee-ai/SuperNova-Mediusarcee-ai59.129
  12. spow12/MK_Nemo_12Bspow1259.104

kornat_common · acc_norm,none

openKoLLM · Open Ko-LLM Leaderboard · kornat_common · acc_norm,none · percent · Higher is better

  1. chihoonlee10/T3Q-ko-solar-dpo-v7.0chihoonlee1050.766%
  2. MoaData/Myrrh_solar_10.7b_3.0moadata48.569%
  3. tanitan2/SOLAR-10.7B-Instruct-T3Q-ko-solar-dpo-tiestanitan244.391%
  4. tanitan2/SOLAR-10.7B-Instruct-STOCK_SOLAR-tiestanitan244.125%
  5. DopeorNope/Dear_My_best_Friends-13Bdopeornope40.446%
  6. Edentns/DataVortexS-10.7B-dpo-v1.10edentns39.947%
  7. freewheelin/free-solar-evo-v0.13freewheelin39.814%
  8. lemon-mint/gemma-ko-7b-instruct-v0.50lemon-mint39.547%
  9. Kukedlc/NeuralSynthesis-7B-v0.1kukedlc38.898%
  10. Gunulhona/Gemma-Ko-Mergegunulhona38.665%
  11. freewheelin/free-solar-evo-v0.11freewheelin38.465%
  12. Edentns/DataVortexS-10.7B-dpo-v1.7edentns38.199%

kornat_helpful · acc_norm,none

openKoLLM · Open Ko-LLM Leaderboard · kornat_helpful · acc_norm,none · percent · Higher is better

  1. tanliboy/lambda-qwen2.5-14b-dpo-testtanliboy66.4%
  2. v000000/Qwen2.5-Lumen-14Bv00000060.9%
  3. Qwen/Qwen2.5-14B-Instructqwen60.65%
  4. Lambent/qwen2.5-reinstruct-alternate-lumen-14Blambent60.475%
  5. v000000/Qwen2.5-14B-Gutenberg-1e-Deltav00000060.2%
  6. v000000/Qwen2.5-14B-Gutenberg-Instruct-Slerpenov00000060.05%
  7. rombodawg/Rombos-LLM-V2.6-Qwen-14brombodawg59.2%
  8. arcee-ai/SuperNova-Mediusarcee-ai58.625%
  9. huihui-ai/Qwen2.5-7B-Instruct-abliteratedhuihui-ai56.275%
  10. princeton-nlp/Llama-3-Base-8B-SFT-RDPOprinceton-nlp56.225%
  11. princeton-nlp/Llama-3-Instruct-8B-SimPO-v0.2princeton-nlp55.95%
  12. Magpie-Align/Llama-3.1-8B-Magpie-Align-v0.1magpie-align55.625%

kornat_social · A-SVA,none

openKoLLM · Open Ko-LLM Leaderboard · kornat_social · A-SVA,none · percent · Higher is better

  1. princeton-nlp/Llama-3-Instruct-8B-RDPO-v0.2princeton-nlp52.779%
  2. princeton-nlp/Llama-3-8B-ProLong-64k-Instructprinceton-nlp52.768%
  3. princeton-nlp/Llama-3-8B-ProLong-512k-Instructprinceton-nlp52.714%
  4. princeton-nlp/Llama-3-Instruct-8B-DPO-v0.2princeton-nlp52.672%
  5. princeton-nlp/Llama-3-8B-ProLong-64k-Baseprinceton-nlp52.384%
  6. princeton-nlp/Llama-3-Instruct-8B-SimPOprinceton-nlp52.242%
  7. princeton-nlp/Llama-3-8B-ProLong-512k-Baseprinceton-nlp52.233%
  8. arcee-ai/Llama-3.1-SuperNova-Litearcee-ai52.12%
  9. princeton-nlp/Llama-3-Instruct-8B-DPOprinceton-nlp52.113%
  10. Magpie-Align/Llama-3-8B-Magpie-Align-SFT-v0.3magpie-align52.088%
  11. Joseph717171/Llama-3.1-SuperNova-8B-Lite_TIES_with_Basejoseph71717152.078%
  12. hyemijo/omed-llama3.1-8bhyemijo52.034%

ko_ifeval · inst_level_strict_acc,none

openKoLLM · Open Ko-LLM Leaderboard · ko_ifeval · inst_level_strict_acc,none · percent · Higher is better

  1. ENERGY-DRINK-LOVE/rtzr_dpo-v4-hqenergy-drink-love67.953%
  2. Qwen: Qwen2.5 7B Instructqwen67.656%
  3. kayfour/T3Q-Qwen2.5-7B-it-KOR-Safekayfour67.507%
  4. Gunulhona/Gemma-Ko-Mergegunulhona59.644%
  5. vicgalle/Configurable-Llama-3.1-8B-Instructvicgalle59.644%
  6. v000000/L3.1-Niitorm-8B-DPO-t0.0001v00000057.567%
  7. lightblue/suzume-llama-3-8B-multilingual-orpo-borda-top25lightblue56.973%
  8. mlabonne/Daredevil-8B-abliteratedmlabonne56.528%
  9. Eurdem/Defne-llama3.1-8Beurdem55.638%
  10. lightblue/suzume-llama-3-8B-multilingual-orpo-borda-top75lightblue55.341%
  11. EpistemeAI/Fireball-Alpaca-Llama3.1.07-8B-Philos-Math-KTO-betaepistemeai55.193%
  12. vicgalle/Configurable-Llama-3-8B-v0.3vicgalle55.193%

kornat_helpful · acc_norm,none

openKoLLM · Open Ko-LLM Leaderboard · kornat_helpful · acc_norm,none · percent · Higher is better

  1. F24/llama-2-koen-orca-mini-platypus2-math-13bf24100%
  2. chihoonlee10/T3Q-ko-solar-dpo-v7.0chihoonlee1082.2%
  3. MoaData/Myrrh_solar_10.7b_3.0moadata82.125%
  4. tanitan2/SOLAR-10.7B-Instruct-T3Q-ko-solar-dpo-tiestanitan279.475%
  5. tanitan2/SOLAR-10.7B-Instruct-STOCK_SOLAR-tiestanitan279.4%
  6. freewheelin/free-solar-evo-v0.1freewheelin77.525%
  7. freewheelin/free-solar-evo-v0.13freewheelin76.8%
  8. freewheelin/free-solar-evo-v0.11freewheelin75.375%
  9. chihoonlee10/T3Q-ko-solar-dpo-v3.0chihoonlee1072.05%
  10. chihoonlee10/T3Q-ko-solar-dpo-v1.0chihoonlee1069.875%
  11. hwkwon/S-SOLAR-10.7B-v1.4hwkwon69.6%
  12. hkss/hk-SOLAR-10.7B-v1.4hkss69.45%

ko_eqbench · percent_parseable,none

openKoLLM · Open Ko-LLM Leaderboard · ko_eqbench · percent_parseable,none · percent · Higher is better

  1. 12thD/ko-Llama-3-8B-sft-v0.112thd89.474%
  2. 12thD/ko-Llama-3-8B-sft-v0.312thd89.474%
  3. AIdenU/SOLAR-10.7b-ko-Y24_v1.0aidenu89.474%
  4. lcw99/llama-3-8b-it-kor-extented-changlcw9989.474%
  5. OpenBuddy/openbuddy-llama3-8b-v21.1-8kopenbuddy89.474%
  6. PathFinderKR/Waktaverse-Llama-3-KO-8B-Instructpathfinderkr89.474%
  7. richard-park/llama3-deepspeed-v1.0richard-park89.474%
  8. SeaLLMs/SeaLLM-7B-v2.5seallms89.474%
  9. sinjy1203/EEVE-Korean-Instruct-10.8B-v1.0-Grade-Retrievalsinjy120389.474%
  10. T3Q-LLM/T3Q-LLM-sft1.0-dpo1.0t3q-llm89.474%
  11. T3Q-LLM/T3Q-LLM-solar10.8-sft-v1.0t3q-llm89.474%
  12. yanolja/Bookworm-10.7B-v0.4-DPOyanolja89.474%

ko_gpqa_diamond_zeroshot · acc_norm,none

openKoLLM · Open Ko-LLM Leaderboard · ko_gpqa_diamond_zeroshot · acc_norm,none · percent · Higher is better

  1. T3Q-LLM/T3Q-LLM1-sft1.0-dpo1.0t3q-llm31.313%
  2. ai-human-lab/EEVE-Korean-10.8B-DPO-v1.0ai-human-lab30.303%
  3. gwonny/nox-solar-10.7b-v4-kolon-all-10gwonny29.798%
  4. jungyuko/DAVinCI-42dot_LLM-PLM-1.3B-v1.5.3jungyuko29.798%
  5. MoaData/Myrrh_solar_10.7b_3.0moadata29.798%
  6. Alphacode-AI/Alphallama3-8B_v2alphacode-ai29.293%
  7. DopeorNope/SOLAR_C-v1-10.7Bdopeornope29.293%
  8. freewheelin/free-solar-evo-v0.13freewheelin29.293%
  9. metterian/llama-2-ko-7b-ptmetterian29.293%
  10. tanitan2/SOLAR-10.7B-Instruct-T3Q-ko-solar-dpo-tiestanitan229.293%
  11. v000000/L3.1-Niitorm-8B-DPO-t0.0001v00000029.293%
  12. Alphacode-AI/Alphacode-MALI-11Balphacode-ai28.788%

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace