Berkeley Function Calling Leaderboard
also referred to as: bfcl
8 statements across 3 episodes · 2 bullish · 2 bearish · 3 people on the record · first statement Sep 17, 2024 by Michelle Pokrass · said 24 times in 6 episodes since 2024 · across every show →
Mentions by year
brought up most by Michelle Pokrass (8), Sam Julien (7), Eugene Yan (4), Shawn Wang (3), Pratik Bhavsar (2)
tap a year for its mentions
2025 4 mentions in 1 episode
2024 20 mentions in 5 episodes 4 per episode
- [Paper Club] Berkeley Function Calling Paper Club! — Sam Julien, Writer
- Building AGI with OpenAI's Structured Outputs API
- [Paper Club] Who Validates the Validators? Aligning LLM-Judges with Humans (w/ Eugene Yan)
- Building AGI in Real Time (OpenAI Dev Day 2024)
- Agents @ Work: Dust.tt — with Stanislas Polu
- every mention in 2024, scene by scene →
Everything said about Berkeley Function Calling Leaderboard, oldest first
Sep 17, 2024 negative
Function calling benchmarks like BFCL are largely saturated
“I find that a lot of these evals are mostly saturated, like for BFCL. All the models are near, near the top. Already. And kind of the errors are more, I would say like just differences in default behaviors. I think most of the models on the leaderboard can kin…”
Oct 5, 2024 positive
Oct 5, 2024 neutral
Oct 5, 2024 positive
BFCL v1 introduced AST-based evaluation for executable functions
“And then the main, like, I think sort of the main innovation of this first version of the leaderboard was that it used this, it introduced this like abstract syntax tree. Evaluation where it would look at the executable functions and evaluate them”
Oct 5, 2024 neutral
Jul 14, 2025 negative
Berkeley Function Calling Leaderboard is unsuitable for realistic agent evaluation
“This BFCL if you just open and check it you would be like, okay, this is okay, but you don't want to use it for realistic evaluation because it's like some math. Problem statement is given. Some simple tool is given just like, let's say some exponential tool, …”