Open Persian LLM Leaderboard

Indexed results from Open Persian LLM Leaderboard. Each ranking retains its own metric and evaluation conditions.

AUT Multiple Choice Persian

openPersianLLM · Open Persian LLM Leaderboard · AUT Multiple Choice Persian · percent · Higher is better

  1. CohereForAI/aya-expanse-32bcohereforai62.33%
  2. CohereForAI/aya-23-35Bcohereforai61.14%
  3. CohereForAI/aya-expanse-8bcohereforai58.38%
  4. CohereForAI/aya-23-8Bcohereforai52.86%

ARC Easy

openPersianLLM · Open Persian LLM Leaderboard · ARC Easy · percent · Higher is better

  1. CohereForAI/aya-expanse-32bcohereforai93.1%
  2. CohereForAI/aya-23-35Bcohereforai87.93%
  3. CohereForAI/aya-23-8Bcohereforai80.46%
  4. CohereForAI/aya-expanse-8bcohereforai79.6%

Part Multiple Choice

openPersianLLM · Open Persian LLM Leaderboard · Part Multiple Choice · percent · Higher is better

  1. Meta: Llama 3.3 70B Instructmeta-llama51.42%
  2. Google: Gemma 3 27Bgoogle48.56%
  3. Qwen/QwQ-32B-Previewqwen46.64%
  4. Qwen/Qwen2.5-32B-Instructqwen46.06%
  5. Google: Gemma 2 27Bgoogle46.03%
  6. Google: Gemma 3 12Bgoogle43.52%
  7. google/gemma-2-9b-itgoogle42.7%
  8. Qwen: Qwen2.5 7B Instructqwen36.72%
  9. Qwen2-7B-Instructqwen35.9%
  10. Llama-3-1-8Bmeta-llama35.52%
  11. NousResearch/Hermes-3-Llama-3.1-8Bnousresearch35.01%
  12. meta-llama/Meta-Llama-3-8B-Instructmeta-llama34.99%

Part Multiple Choice

openPersianLLM · Open Persian LLM Leaderboard · Part Multiple Choice · percent · Higher is better

  1. CohereForAI/aya-expanse-32bcohereforai43.36%
  2. CohereForAI/aya-23-35Bcohereforai36.79%
  3. CohereForAI/aya-expanse-8bcohereforai34.91%
  4. CohereForAI/aya-23-8Bcohereforai32.82%

ARC Challenge

openPersianLLM · Open Persian LLM Leaderboard · ARC Challenge · percent · Higher is better

  1. Google: Gemma 3 27Bgoogle90.6%
  2. Meta: Llama 3.3 70B Instructmeta-llama89.93%
  3. Qwen/QwQ-32B-Previewqwen87.24%
  4. Google: Gemma 2 27Bgoogle85.91%
  5. Qwen/Qwen2.5-32B-Instructqwen85.91%
  6. google/gemma-2-9b-itgoogle84.56%
  7. Google: Gemma 3 12Bgoogle81.21%
  8. Qwen: Qwen2.5 7B Instructqwen69.13%
  9. Qwen2-7B-Instructqwen68.46%
  10. Google: Gemma 3 4Bgoogle63.76%
  11. NousResearch/Hermes-3-Llama-3.1-8Bnousresearch58.39%
  12. Gemma2-2b-itgoogle57.72%

MMLU Pro

openPersianLLM · Open Persian LLM Leaderboard · MMLU Pro · percent · Higher is better

  1. CohereForAI/aya-expanse-32bcohereforai31.03%
  2. CohereForAI/aya-23-35Bcohereforai25.54%
  3. CohereForAI/aya-expanse-8bcohereforai25.06%
  4. CohereForAI/aya-23-8Bcohereforai18.62%

MMLU Pro

openPersianLLM · Open Persian LLM Leaderboard · MMLU Pro · percent · Higher is better

  1. Meta: Llama 3.3 70B Instructmeta-llama43.67%
  2. Google: Gemma 3 27Bgoogle40.1%
  3. Qwen/Qwen2.5-32B-Instructqwen38.19%
  4. Qwen/QwQ-32B-Previewqwen37.94%
  5. Google: Gemma 2 27Bgoogle36.28%
  6. google/gemma-2-9b-itgoogle31.74%
  7. Google: Gemma 3 12Bgoogle29.36%
  8. meta-llama/Meta-Llama-3-8B-Instructmeta-llama25.54%
  9. Llama-3-1-8Bmeta-llama24.1%
  10. Qwen2-7B-Instructqwen23.87%
  11. Qwen: Qwen2.5 7B Instructqwen21.96%
  12. lightblue/suzume-llama-3-8B-multilinguallightblue21.95%

ARC Easy

openPersianLLM · Open Persian LLM Leaderboard · ARC Easy · percent · Higher is better

  1. Google: Gemma 2 27Bgoogle95.98%
  2. Google: Gemma 3 27Bgoogle95.69%
  3. Meta: Llama 3.3 70B Instructmeta-llama95.4%
  4. Google: Gemma 3 12Bgoogle93.39%
  5. google/gemma-2-9b-itgoogle93.1%
  6. Qwen/QwQ-32B-Previewqwen91.95%
  7. Qwen/Qwen2.5-32B-Instructqwen90.8%
  8. Qwen: Qwen2.5 7B Instructqwen79.02%
  9. Qwen2-7B-Instructqwen77.3%
  10. Google: Gemma 3 4Bgoogle77.01%
  11. NousResearch/Hermes-3-Llama-3.1-8Bnousresearch77.01%
  12. Llama-3-1-8Bmeta-llama75.28%

ARC Challenge

openPersianLLM · Open Persian LLM Leaderboard · ARC Challenge · percent · Higher is better

  1. CohereForAI/aya-expanse-32bcohereforai79.87%
  2. CohereForAI/aya-23-35Bcohereforai72.48%
  3. CohereForAI/aya-expanse-8bcohereforai70.47%
  4. CohereForAI/aya-23-8Bcohereforai64.43%

AUT Multiple Choice Persian

openPersianLLM · Open Persian LLM Leaderboard · AUT Multiple Choice Persian · percent · Higher is better

  1. Meta: Llama 3.3 70B Instructmeta-llama71.4%
  2. Google: Gemma 3 27Bgoogle64.3%
  3. Google: Gemma 2 27Bgoogle63.12%
  4. google/gemma-2-9b-itgoogle62.33%
  5. Qwen/Qwen2.5-32B-Instructqwen61.34%
  6. Qwen/QwQ-32B-Previewqwen60.15%
  7. Google: Gemma 3 12Bgoogle57%
  8. meta-llama/Meta-Llama-3-8B-Instructmeta-llama53.85%
  9. Llama-3-1-8Bmeta-llama53.45%
  10. Qwen: Qwen2.5 7B Instructqwen52.66%
  11. NousResearch/Hermes-3-Llama-3.1-8Bnousresearch52.46%
  12. Qwen2-7B-Instructqwen51.68%

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace