SambaNova ToolBench
Indexed results from SambaNova ToolBench. Each ranking retains its own metric and evaluation conditions.
VirtualHome
toolBench · SambaNova ToolBench · VirtualHome · score · Higher is better
- codellama/CodeLlama-34b-hfcodellama24.6
- codellama/CodeLlama-34b-Instruct-hfcodellama24.3
- codellama/CodeLlama-13b-Instruct-hfcodellama22.6
- codellama/CodeLlama-13b-hfcodellama22.3
- codellama/CodeLlama-7b-hfcodellama22
- codellama/CodeLlama-7b-Instruct-hfcodellama22
- codellama/CodeLlama-13b-Python-hfcodellama21.9
- huggyllama/llama-30bhuggyllama21.7
- codellama/CodeLlama-7b-Python-hfcodellama21.3
- codellama/CodeLlama-34b-Python-hfcodellama21.2
- huggyllama/llama-65bhuggyllama20.3
- bigcode/starcoderbigcode19.4
Avg.
toolBench · SambaNova ToolBench · Avg. · percent · Higher is better
- codellama/CodeLlama-34b-Instruct-hfcodellama64.8%
- codellama/CodeLlama-34b-hfcodellama62.9%
- codellama/CodeLlama-13b-Instruct-hfcodellama60.5%
- codellama/CodeLlama-34b-Python-hfcodellama59.2%
- codellama/CodeLlama-13b-hfcodellama56.9%
- codellama/CodeLlama-13b-Python-hfcodellama56.3%
- huggyllama/llama-65bhuggyllama55.6%
- bigcode/starcoderbasebigcode52.2%
- codellama/CodeLlama-7b-Python-hfcodellama52.2%
- codellama/CodeLlama-7b-Instruct-hfcodellama50.5%
- bigcode/starcoderbigcode49.7%
- huggyllama/llama-30bhuggyllama49.6%
Tabletop
toolBench · SambaNova ToolBench · Tabletop · percent · Higher is better
- codellama/CodeLlama-13b-Instruct-hfcodellama57.6%
- codellama/CodeLlama-34b-hfcodellama51.3%
- codellama/CodeLlama-34b-Instruct-hfcodellama47.5%
- codellama/CodeLlama-13b-Python-hfcodellama41.5%
- codellama/CodeLlama-13b-hfcodellama41%
- huggyllama/llama-30bhuggyllama34.3%
- codellama/CodeLlama-34b-Python-hfcodellama33.3%
- huggyllama/llama-65bhuggyllama30.5%
- codellama/CodeLlama-7b-Python-hfcodellama22.9%
- bigcode/starcoderbigcode21.9%
- bigcode/starcoderbasebigcode17.1%
- huggyllama/llama-13bhuggyllama17.1%
The Cat API
toolBench · SambaNova ToolBench · The Cat API · percent · Higher is better
- codellama/CodeLlama-7b-Instruct-hfcodellama94%
- codellama/CodeLlama-13b-Python-hfcodellama92.4%
- codellama/CodeLlama-7b-hfcodellama92%
- codellama/CodeLlama-13b-Instruct-hfcodellama89.6%
- codellama/CodeLlama-34b-Python-hfcodellama88.4%
- codellama/CodeLlama-7b-Python-hfcodellama88%
- codellama/CodeLlama-13b-hfcodellama86.8%
- bigcode/starcoderbasebigcode86%
- codellama/CodeLlama-34b-Instruct-hfcodellama86%
- bigscience/bloomzbigscience85%
- codellama/CodeLlama-34b-hfcodellama85%
- huggyllama/llama-30bhuggyllama84%
Open Weather
toolBench · SambaNova ToolBench · Open Weather · percent · Higher is better
- codellama/CodeLlama-13b-Instruct-hfcodellama98%
- codellama/CodeLlama-34b-hfcodellama96.4%
- codellama/CodeLlama-13b-hfcodellama96%
- codellama/CodeLlama-34b-Instruct-hfcodellama94.3%
- codellama/CodeLlama-13b-Python-hfcodellama93%
- codellama/CodeLlama-34b-Python-hfcodellama91.1%
- bigcode/starcoderbigcode91%
- bigcode/starcoderbasebigcode90%
- codellama/CodeLlama-7b-Instruct-hfcodellama90%
- huggyllama/llama-65bhuggyllama90%
- codellama/CodeLlama-7b-hfcodellama86%
- codellama/CodeLlama-7b-Python-hfcodellama83%
Trip Booking
toolBench · SambaNova ToolBench · Trip Booking · percent · Higher is better
- codellama/CodeLlama-34b-Instruct-hfcodellama89.2%
- codellama/CodeLlama-34b-hfcodellama88.3%
- codellama/CodeLlama-34b-Python-hfcodellama85.8%
- codellama/CodeLlama-13b-Instruct-hfcodellama72.5%
- codellama/CodeLlama-13b-hfcodellama70.8%
- codellama/CodeLlama-7b-Python-hfcodellama68.3%
- huggyllama/llama-65bhuggyllama65.8%
- bigcode/starcoderbasebigcode63.3%
- codellama/CodeLlama-7b-hfcodellama63.3%
- codellama/CodeLlama-13b-Python-hfcodellama62.5%
- codellama/CodeLlama-7b-Instruct-hfcodellama61.7%
- bigcode/starcoderbigcode51.7%
WebShop Long
toolBench · SambaNova ToolBench · WebShop Long · score · Higher is better
- bigcode/starcoderbasebigcode5.8
- bigcode/starcoderbigcode2.6
- bigscience/bloomzbigscience0
- chavinlo/gpt4-x-alpacachavinlo0
- codellama/CodeLlama-13b-Python-hfcodellama0
- EleutherAI/pythia-12beleutherai0
- huggyllama/llama-13bhuggyllama0
- huggyllama/llama-30bhuggyllama0
- huggyllama/llama-65bhuggyllama0
- stabilityai/stablelm-base-alpha-3bstabilityai0
- stabilityai/stablelm-base-alpha-7bstabilityai0
- stabilityai/stablelm-tuned-alpha-3bstabilityai0
Google Sheets
toolBench · SambaNova ToolBench · Google Sheets · percent · Higher is better
- codellama/CodeLlama-34b-hfcodellama64.3%
- codellama/CodeLlama-34b-Instruct-hfcodellama61.1%
- codellama/CodeLlama-34b-Python-hfcodellama55.9%
- codellama/CodeLlama-13b-hfcodellama51.3%
- codellama/CodeLlama-13b-Python-hfcodellama50.8%
- codellama/CodeLlama-7b-Python-hfcodellama49.1%
- codellama/CodeLlama-13b-Instruct-hfcodellama49%
- bigcode/starcoderbigcode48%
- bigcode/starcoderbasebigcode42.9%
- codellama/CodeLlama-7b-Instruct-hfcodellama41.3%
- codellama/CodeLlama-7b-hfcodellama38.1%
- huggyllama/llama-30bhuggyllama37.1%
WebShop Short
toolBench · SambaNova ToolBench · WebShop Short · score · Higher is better
- huggyllama/llama-65bhuggyllama41.2
- huggyllama/llama-30bhuggyllama30.6
- huggyllama/llama-13bhuggyllama27.6
- codellama/CodeLlama-34b-Instruct-hfcodellama26
- bigcode/starcoderbasebigcode23.1
- codellama/CodeLlama-13b-Instruct-hfcodellama9.7
- EleutherAI/gpt-neox-20beleutherai8.8
- codellama/CodeLlama-34b-Python-hfcodellama6.5
- codellama/CodeLlama-34b-hfcodellama5.5
- EleutherAI/pythia-1.4beleutherai5.2
- chavinlo/gpt4-x-alpacachavinlo2.7
- codellama/CodeLlama-13b-Python-hfcodellama2.4
Home Search
toolBench · SambaNova ToolBench · Home Search · percent · Higher is better
- codellama/CodeLlama-34b-Python-hfcodellama91%
- codellama/CodeLlama-34b-Instruct-hfcodellama90%
- codellama/CodeLlama-34b-hfcodellama88%
- codellama/CodeLlama-13b-hfcodellama87%
- codellama/CodeLlama-13b-Python-hfcodellama86%
- codellama/CodeLlama-13b-Instruct-hfcodellama85%
- huggyllama/llama-65bhuggyllama84%
- codellama/CodeLlama-7b-Python-hfcodellama83%
- bigcode/starcoderbigcode82%
- bigcode/starcoderbasebigcode79%
- codellama/CodeLlama-7b-Instruct-hfcodellama78%
- codellama/CodeLlama-7b-hfcodellama74%
Models
LoadingBenchmarks
Explore every benchmark
LoadingThese charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.