Subquadratic LLM Leaderboard

Indexed results from Subquadratic LLM Leaderboard. Each ranking retains its own metric and evaluation conditions.

Winogrande

subquadraticLLM · Subquadratic LLM Leaderboard · Winogrande · percent · Higher is better

  1. google/recurrentgemma-2bgoogle68.51%
  2. RWKV/rwkv-raven-14brwkv66.69%
  3. RWKV/rwkv-4-14b-pilerwkv65.43%
  4. google/recurrentgemma-2b-itgoogle64.17%
  5. RWKV/rwkv-raven-7brwkv62.9%
  6. RWKV/rwkv-4-7b-pilerwkv62.35%
  7. devingulliver/llama-pile-350bdevingulliver58.48%
  8. RWKV/rwkv-4-3b-pilerwkv58.33%
  9. RWKV/rwkv-raven-3brwkv57.46%
  10. RWKV/rwkv-raven-1b5rwkv53.91%
  11. RWKV/rwkv-4-1b5-pilerwkv53.83%

HellaSwag

subquadraticLLM · Subquadratic LLM Leaderboard · HellaSwag · percent · Higher is better

  1. RWKV/rwkv-raven-14brwkv71.25%
  2. RWKV/rwkv-4-14b-pilerwkv71.07%
  3. RWKV/rwkv-raven-7brwkv66.48%
  4. RWKV/rwkv-4-7b-pilerwkv66.31%
  5. RWKV/rwkv-raven-3brwkv59.78%
  6. RWKV/rwkv-4-3b-pilerwkv59.66%
  7. google/recurrentgemma-2bgoogle56.89%
  8. devingulliver/llama-pile-350bdevingulliver56.6%
  9. google/recurrentgemma-2b-itgoogle56.26%
  10. RWKV/rwkv-raven-1b5rwkv52.6%
  11. RWKV/rwkv-4-1b5-pilerwkv52.25%

ARC

subquadraticLLM · Subquadratic LLM Leaderboard · ARC · percent · Higher is better

  1. RWKV/rwkv-raven-14brwkv44.62%
  2. RWKV/rwkv-4-14b-pilerwkv44.45%
  3. RWKV/rwkv-4-7b-pilerwkv39.68%
  4. RWKV/rwkv-raven-7brwkv39.42%
  5. RWKV/rwkv-raven-3brwkv36.69%
  6. RWKV/rwkv-4-3b-pilerwkv36.01%
  7. devingulliver/llama-pile-350bdevingulliver33.19%
  8. RWKV/rwkv-4-1b5-pilerwkv31.83%
  9. RWKV/rwkv-raven-1b5rwkv31.83%
  10. google/recurrentgemma-2bgoogle31.4%
  11. google/recurrentgemma-2b-itgoogle30.97%

GSM8K

subquadraticLLM · Subquadratic LLM Leaderboard · GSM8K · percent · Higher is better

  1. google/recurrentgemma-2bgoogle16.15%
  2. google/recurrentgemma-2b-itgoogle10.08%
  3. RWKV/rwkv-raven-14brwkv2.12%
  4. devingulliver/llama-pile-350bdevingulliver0.76%
  5. RWKV/rwkv-4-7b-pilerwkv0.76%
  6. RWKV/rwkv-4-3b-pilerwkv0.68%
  7. RWKV/rwkv-raven-3brwkv0.45%
  8. RWKV/rwkv-4-14b-pilerwkv0.38%
  9. RWKV/rwkv-raven-7brwkv0.3%
  10. RWKV/rwkv-4-1b5-pilerwkv0%
  11. RWKV/rwkv-raven-1b5rwkv0%

MMLU

subquadraticLLM · Subquadratic LLM Leaderboard · MMLU · percent · Higher is better

  1. google/recurrentgemma-2b-itgoogle40.87%
  2. google/recurrentgemma-2bgoogle34.61%
  3. RWKV/rwkv-4-14b-pilerwkv26.12%
  4. RWKV/rwkv-raven-1b5rwkv25.96%
  5. RWKV/rwkv-raven-14brwkv25.92%
  6. RWKV/rwkv-4-1b5-pilerwkv25.77%
  7. RWKV/rwkv-4-7b-pilerwkv24.96%
  8. RWKV/rwkv-raven-3brwkv24.87%
  9. RWKV/rwkv-4-3b-pilerwkv24.67%
  10. devingulliver/llama-pile-350bdevingulliver24.66%
  11. RWKV/rwkv-raven-7brwkv23.64%

Average

subquadraticLLM · Subquadratic LLM Leaderboard · Average · percent · Higher is better

  1. RWKV/rwkv-raven-14brwkv42.09%
  2. google/recurrentgemma-2b-itgoogle40.86%
  3. google/recurrentgemma-2bgoogle40.44%
  4. RWKV/rwkv-4-14b-pilerwkv39.92%
  5. RWKV/rwkv-raven-7brwkv38.55%
  6. RWKV/rwkv-4-7b-pilerwkv37.95%
  7. RWKV/rwkv-raven-3brwkv35.81%
  8. RWKV/rwkv-4-3b-pilerwkv35.25%
  9. devingulliver/llama-pile-350bdevingulliver35%
  10. RWKV/rwkv-raven-1b5rwkv33.56%
  11. RWKV/rwkv-4-1b5-pilerwkv33.25%

TruthfulQA

subquadraticLLM · Subquadratic LLM Leaderboard · TruthfulQA · percent · Higher is better

  1. google/recurrentgemma-2b-itgoogle42.81%
  2. RWKV/rwkv-raven-14brwkv41.93%
  3. RWKV/rwkv-raven-7brwkv38.56%
  4. RWKV/rwkv-raven-1b5rwkv37.09%
  5. devingulliver/llama-pile-350bdevingulliver36.28%
  6. RWKV/rwkv-4-1b5-pilerwkv35.8%
  7. RWKV/rwkv-raven-3brwkv35.6%
  8. google/recurrentgemma-2bgoogle35.1%
  9. RWKV/rwkv-4-7b-pilerwkv33.65%
  10. RWKV/rwkv-4-3b-pilerwkv32.14%
  11. RWKV/rwkv-4-14b-pilerwkv32.04%

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace