BFCL
Indexed results from BFCL. Each ranking retains its own metric and evaluation conditions.
BFCL Irrelevance Detection
bfcl · BFCL · Irrelevance detection · percent
- nvidia/llama-3.1-nemotron-ultra-253b-v1nvidia100%
- mistralai/Ministral-8B-Instruct-2410mistralai100%
- claude-haiku-4-5@20251001anthropic95.29%
- Anthropic: Claude Sonnet 4.5anthropic95.03%
- Google: Gemini 2.5 Flashgoogle93.67%
- Google: Gemini 2.5 Flash Litegoogle93.33%
- DeepSeek: DeepSeek V3.2 Expdeepseek93.18%
- Google: Gemini 2.5 Flash Litegoogle92.5%
- Google: Gemini 2.5 Flashgoogle91.09%
- OpenAI: GPT-5 Miniopenai91.01%
- claude-opus-4-5@20251101anthropic90.75%
- OpenAI: GPT-5 Nanoopenai89.1%
BFCL Live
bfcl · BFCL · Live accuracy · percent
- gemini-3-pro-previewgoogle83.12%
- Qwen: Qwen3 32Bqwen82.01%
- Qwen: Qwen3 32Bqwen82.01%
- gemini-3-pro-previewgoogle81.72%
- Anthropic: Claude Sonnet 4.5anthropic81.13%
- Z.ai: GLM 4.6z-ai80.9%
- Qwen: Qwen3 14Bqwen80.01%
- claude-opus-4-5@20251101anthropic79.79%
- Nanbeige/Nanbeige4-3B-Thinking-2511nanbeige79.42%
- Qwen: Qwen3 14Bqwen79.35%
- OpenAI: GPT-4.1openai78.9%
- claude-haiku-4-5@20251001anthropic78.68%
BFCL Memory
bfcl · BFCL · Memory accuracy · percent
- claude-opus-4-5@20251101anthropic73.76%
- Anthropic: Claude Sonnet 4.5anthropic64.95%
- gemini-3-pro-previewgoogle61.72%
- Z.ai: GLM 4.6z-ai55.7%
- gemini-3-pro-previewgoogle54.84%
- claude-haiku-4-5@20251001anthropic54.41%
- DeepSeek: DeepSeek V3.2 Expdeepseek54.19%
- Grok 4.1 Fast Reasoningspacexai53.98%
- OpenAI: o3openai51.83%
- OpenAI: o3openai47.31%
- OpenAI gpt-5.2 · azure-openai v2025-12-11openai45.81%
- OpenAI: GPT-5 Miniopenai44.3%
BFCL Multi Turn
bfcl · BFCL · Multi-turn accuracy · percent
- Salesforce/xLAM-2-32b-fc-rsalesforce69.5%
- claude-opus-4-5@20251101anthropic68.38%
- Z.ai: GLM 4.6z-ai68%
- gemini-3-pro-previewgoogle63.12%
- OpenAI: o3openai62.25%
- Anthropic: Claude Sonnet 4.5anthropic61.37%
- gemini-3-pro-previewgoogle60.75%
- Grok 4.1 Fast Reasoningspacexai58.87%
- Salesforce/xLAM-2-3b-fc-rsalesforce58.38%
- claude-haiku-4-5@20251001anthropic53.62%
- Nanbeige/Nanbeige4-3B-Thinking-2511nanbeige51.12%
- Qwen: Qwen3 32Bqwen47.87%
BFCL Non-Live AST
bfcl · BFCL · Non-live AST accuracy · percent
- gemini-3-pro-previewgoogle90.65%
- Qwen: Qwen3 32Bqwen90.27%
- claude-opus-4-5@20251101anthropic89.65%
- Salesforce/xLAM-2-32b-fc-rsalesforce89.6%
- Qwen: Qwen3 14Bqwen89.46%
- Meta: Llama 4 Scoutmeta-llama89.38%
- Qwen: Qwen3 30B A3B Instruct 2507qwen88.92%
- Qwen: Qwen3 32Bqwen88.77%
- OpenAI: GPT-4.1openai88.69%
- Anthropic: Claude Sonnet 4.5anthropic88.65%
- claude-opus-4-5@20251101anthropic88.58%
- Grok 4.1 Fast Reasoningspacexai88.27%
BFCL Overall
bfcl · BFCL · Overall accuracy · percent
- claude-opus-4-5@20251101anthropic77.47%
- Anthropic: Claude Sonnet 4.5anthropic73.24%
- gemini-3-pro-previewgoogle72.51%
- Z.ai: GLM 4.6z-ai72.38%
- Grok 4.1 Fast Reasoningspacexai69.57%
- claude-haiku-4-5@20251001anthropic68.7%
- gemini-3-pro-previewgoogle68.14%
- OpenAI: o3openai63.05%
- Grok 4.1 Fast Non-Reasoningspacexai58.29%
- DeepSeek: DeepSeek V3.2 Expdeepseek56.73%
- Google: Gemini 2.5 Flashgoogle56.24%
- OpenAI gpt-5.2 · azure-openai v2025-12-11openai55.87%
BFCL Relevance Detection
bfcl · BFCL · Relevance detection · percent
- tiiuae/Falcon3-7B-Instructtiiuae100%
- OpenAI: GPT-4.1openai100%
- Meta: Llama 4 Scoutmeta-llama100%
- speakleash/Bielik-11B-v2.3-Instructspeakleash93.75%
- DeepSeek: DeepSeek V3.2 Expdeepseek93.75%
- tiiuae/Falcon3-10B-Instructtiiuae93.75%
- OpenAI: GPT-4.1 Nanoopenai93.75%
- OpenAI: GPT-5 Miniopenai93.75%
- OpenAI: GPT-5 Nanoopenai93.75%
- OpenAI: o3openai93.75%
- Qwen: Qwen3 30B A3B Instruct 2507qwen93.75%
- Qwen: Qwen3 32Bqwen93.75%
BFCL Web Search
bfcl · BFCL · Web-search accuracy · percent
- claude-opus-4-5@20251101anthropic84.5%
- claude-haiku-4-5@20251001anthropic83.5%
- Grok 4.1 Fast Reasoningspacexai82.5%
- OpenAI: GPT-5 Miniopenai82%
- Anthropic: Claude Sonnet 4.5anthropic81%
- gemini-3-pro-previewgoogle80%
- Z.ai: GLM 4.6z-ai77.5%
- OpenAI: o3openai77%
- OpenAI gpt-5.2 · azure-openai v2025-12-11openai75.5%
- OpenAI: o4 Mini (batch)openai75.5%
- Grok 4.1 Fast Non-Reasoningspacexai75%
- OpenAI: GPT-5 Nanoopenai72.5%
Models
LoadingBenchmarks
Explore every benchmark
LoadingThese charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.