SambaNova ToolBench

Indexed results from SambaNova ToolBench. Each ranking retains its own metric and evaluation conditions.

VirtualHome

toolBench · SambaNova ToolBench · VirtualHome · score · Higher is better

  1. codellama/CodeLlama-34b-hfcodellama24.6
  2. codellama/CodeLlama-34b-Instruct-hfcodellama24.3
  3. codellama/CodeLlama-13b-Instruct-hfcodellama22.6
  4. codellama/CodeLlama-13b-hfcodellama22.3
  5. codellama/CodeLlama-7b-hfcodellama22
  6. codellama/CodeLlama-7b-Instruct-hfcodellama22
  7. codellama/CodeLlama-13b-Python-hfcodellama21.9
  8. huggyllama/llama-30bhuggyllama21.7
  9. codellama/CodeLlama-7b-Python-hfcodellama21.3
  10. codellama/CodeLlama-34b-Python-hfcodellama21.2
  11. huggyllama/llama-65bhuggyllama20.3
  12. bigcode/starcoderbigcode19.4

Avg.

toolBench · SambaNova ToolBench · Avg. · percent · Higher is better

  1. codellama/CodeLlama-34b-Instruct-hfcodellama64.8%
  2. codellama/CodeLlama-34b-hfcodellama62.9%
  3. codellama/CodeLlama-13b-Instruct-hfcodellama60.5%
  4. codellama/CodeLlama-34b-Python-hfcodellama59.2%
  5. codellama/CodeLlama-13b-hfcodellama56.9%
  6. codellama/CodeLlama-13b-Python-hfcodellama56.3%
  7. huggyllama/llama-65bhuggyllama55.6%
  8. bigcode/starcoderbasebigcode52.2%
  9. codellama/CodeLlama-7b-Python-hfcodellama52.2%
  10. codellama/CodeLlama-7b-Instruct-hfcodellama50.5%
  11. bigcode/starcoderbigcode49.7%
  12. huggyllama/llama-30bhuggyllama49.6%

Tabletop

toolBench · SambaNova ToolBench · Tabletop · percent · Higher is better

  1. codellama/CodeLlama-13b-Instruct-hfcodellama57.6%
  2. codellama/CodeLlama-34b-hfcodellama51.3%
  3. codellama/CodeLlama-34b-Instruct-hfcodellama47.5%
  4. codellama/CodeLlama-13b-Python-hfcodellama41.5%
  5. codellama/CodeLlama-13b-hfcodellama41%
  6. huggyllama/llama-30bhuggyllama34.3%
  7. codellama/CodeLlama-34b-Python-hfcodellama33.3%
  8. huggyllama/llama-65bhuggyllama30.5%
  9. codellama/CodeLlama-7b-Python-hfcodellama22.9%
  10. bigcode/starcoderbigcode21.9%
  11. bigcode/starcoderbasebigcode17.1%
  12. huggyllama/llama-13bhuggyllama17.1%

The Cat API

toolBench · SambaNova ToolBench · The Cat API · percent · Higher is better

  1. codellama/CodeLlama-7b-Instruct-hfcodellama94%
  2. codellama/CodeLlama-13b-Python-hfcodellama92.4%
  3. codellama/CodeLlama-7b-hfcodellama92%
  4. codellama/CodeLlama-13b-Instruct-hfcodellama89.6%
  5. codellama/CodeLlama-34b-Python-hfcodellama88.4%
  6. codellama/CodeLlama-7b-Python-hfcodellama88%
  7. codellama/CodeLlama-13b-hfcodellama86.8%
  8. bigcode/starcoderbasebigcode86%
  9. codellama/CodeLlama-34b-Instruct-hfcodellama86%
  10. bigscience/bloomzbigscience85%
  11. codellama/CodeLlama-34b-hfcodellama85%
  12. huggyllama/llama-30bhuggyllama84%

Open Weather

toolBench · SambaNova ToolBench · Open Weather · percent · Higher is better

  1. codellama/CodeLlama-13b-Instruct-hfcodellama98%
  2. codellama/CodeLlama-34b-hfcodellama96.4%
  3. codellama/CodeLlama-13b-hfcodellama96%
  4. codellama/CodeLlama-34b-Instruct-hfcodellama94.3%
  5. codellama/CodeLlama-13b-Python-hfcodellama93%
  6. codellama/CodeLlama-34b-Python-hfcodellama91.1%
  7. bigcode/starcoderbigcode91%
  8. bigcode/starcoderbasebigcode90%
  9. codellama/CodeLlama-7b-Instruct-hfcodellama90%
  10. huggyllama/llama-65bhuggyllama90%
  11. codellama/CodeLlama-7b-hfcodellama86%
  12. codellama/CodeLlama-7b-Python-hfcodellama83%

Trip Booking

toolBench · SambaNova ToolBench · Trip Booking · percent · Higher is better

  1. codellama/CodeLlama-34b-Instruct-hfcodellama89.2%
  2. codellama/CodeLlama-34b-hfcodellama88.3%
  3. codellama/CodeLlama-34b-Python-hfcodellama85.8%
  4. codellama/CodeLlama-13b-Instruct-hfcodellama72.5%
  5. codellama/CodeLlama-13b-hfcodellama70.8%
  6. codellama/CodeLlama-7b-Python-hfcodellama68.3%
  7. huggyllama/llama-65bhuggyllama65.8%
  8. bigcode/starcoderbasebigcode63.3%
  9. codellama/CodeLlama-7b-hfcodellama63.3%
  10. codellama/CodeLlama-13b-Python-hfcodellama62.5%
  11. codellama/CodeLlama-7b-Instruct-hfcodellama61.7%
  12. bigcode/starcoderbigcode51.7%

WebShop Long

toolBench · SambaNova ToolBench · WebShop Long · score · Higher is better

  1. bigcode/starcoderbasebigcode5.8
  2. bigcode/starcoderbigcode2.6
  3. bigscience/bloomzbigscience0
  4. chavinlo/gpt4-x-alpacachavinlo0
  5. codellama/CodeLlama-13b-Python-hfcodellama0
  6. EleutherAI/pythia-12beleutherai0
  7. huggyllama/llama-13bhuggyllama0
  8. huggyllama/llama-30bhuggyllama0
  9. huggyllama/llama-65bhuggyllama0
  10. stabilityai/stablelm-base-alpha-3bstabilityai0
  11. stabilityai/stablelm-base-alpha-7bstabilityai0
  12. stabilityai/stablelm-tuned-alpha-3bstabilityai0

Google Sheets

toolBench · SambaNova ToolBench · Google Sheets · percent · Higher is better

  1. codellama/CodeLlama-34b-hfcodellama64.3%
  2. codellama/CodeLlama-34b-Instruct-hfcodellama61.1%
  3. codellama/CodeLlama-34b-Python-hfcodellama55.9%
  4. codellama/CodeLlama-13b-hfcodellama51.3%
  5. codellama/CodeLlama-13b-Python-hfcodellama50.8%
  6. codellama/CodeLlama-7b-Python-hfcodellama49.1%
  7. codellama/CodeLlama-13b-Instruct-hfcodellama49%
  8. bigcode/starcoderbigcode48%
  9. bigcode/starcoderbasebigcode42.9%
  10. codellama/CodeLlama-7b-Instruct-hfcodellama41.3%
  11. codellama/CodeLlama-7b-hfcodellama38.1%
  12. huggyllama/llama-30bhuggyllama37.1%

WebShop Short

toolBench · SambaNova ToolBench · WebShop Short · score · Higher is better

  1. huggyllama/llama-65bhuggyllama41.2
  2. huggyllama/llama-30bhuggyllama30.6
  3. huggyllama/llama-13bhuggyllama27.6
  4. codellama/CodeLlama-34b-Instruct-hfcodellama26
  5. bigcode/starcoderbasebigcode23.1
  6. codellama/CodeLlama-13b-Instruct-hfcodellama9.7
  7. EleutherAI/gpt-neox-20beleutherai8.8
  8. codellama/CodeLlama-34b-Python-hfcodellama6.5
  9. codellama/CodeLlama-34b-hfcodellama5.5
  10. EleutherAI/pythia-1.4beleutherai5.2
  11. chavinlo/gpt4-x-alpacachavinlo2.7
  12. codellama/CodeLlama-13b-Python-hfcodellama2.4

Home Search

toolBench · SambaNova ToolBench · Home Search · percent · Higher is better

  1. codellama/CodeLlama-34b-Python-hfcodellama91%
  2. codellama/CodeLlama-34b-Instruct-hfcodellama90%
  3. codellama/CodeLlama-34b-hfcodellama88%
  4. codellama/CodeLlama-13b-hfcodellama87%
  5. codellama/CodeLlama-13b-Python-hfcodellama86%
  6. codellama/CodeLlama-13b-Instruct-hfcodellama85%
  7. huggyllama/llama-65bhuggyllama84%
  8. codellama/CodeLlama-7b-Python-hfcodellama83%
  9. bigcode/starcoderbigcode82%
  10. bigcode/starcoderbasebigcode79%
  11. codellama/CodeLlama-7b-Instruct-hfcodellama78%
  12. codellama/CodeLlama-7b-hfcodellama74%

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace