StarZero All shows Try the podcast clipper →

Sam Julien

Director of Product, CopilotKit · 1 appearance on the record.

computed by AI from the episodes · how this works → · full disclaimer →

operatorauthorexecutive@samjulien ↗LinkedIn ↗samjulien.com ↗

Sam Julien is an AI product and DevRel leader who built the developer relations program and worked as a product manager at Writer, following years leading developer advocacy at Auth0 and Okta. He is the author of Getting Started in Developer Relations and co-author of Agentic GraphRAG.

4statements → 3claims → 2claims resolved → 3.75/5average certainty → 1.25/5average debate potential → 2said about them ↓

1 supported 0 partly supported 1 contradicted 1 not yet assessed how the 3 claims stand · each chip opens the sources

3 assertions · 1 opinion · every statement was checked. The predictions and assertions are the 3 claims: statements the public record can support or contradict. 2 are resolved, 1 is not yet assessed. Everything else (opinions, insights, what ifs, disclosures) can never be settled by the record, so it carries no assessment.

The record, in short

What the tape says about how Sam argues and how the claims held up. Everything they said, and everything said about them, is in the tabs below.

Their most notable supported claim

Assertion Supported
Julien: BFCL v3 shifted from AST to state-based evaluation
“So they really updated their evaluation from the abstract syntax tree to something called state-based evaluation, which we'll, we'll talk about in just a second.”
Sam Julien Oct 5, 2024 ▶ 8:24 [Paper Club] Berkeley Function Calling Paper Club! — Sam Julien, Writer

Their most notable contradicted claim

Assertion Contradicted
Julien: BFCL v1 was the first dedicated LLM function calling benchmark
“So the first one that came out, like I said, in March was really the first of its kind to Make a function calling leaderboard.”
Sam Julien Oct 5, 2024 ▶ 2:10 [Paper Club] Berkeley Function Calling Paper Club! — Sam Julien, Writer

Everything Sam Julien said on Latent Space that made the record, most notable first. Filter by type, assessment or year in the ledger →

Opinion
BFCL v1 introduced AST-based evaluation for executable functions
“And then the main, like, I think sort of the main innovation of this first version of the leaderboard was that it used this, it introduced this like abstract syntax tree. Evaluation where it would look at the executable functions and evaluate them”
Sam Julien Oct 5, 2024 ▶ 3:52 [Paper Club] Berkeley Function Calling Paper Club! — Sam Julien, Writer
Assertion Contradicted
Julien: BFCL v1 was the first dedicated LLM function calling benchmark
“So the first one that came out, like I said, in March was really the first of its kind to Make a function calling leaderboard.”
Sam Julien Oct 5, 2024 ▶ 2:10 [Paper Club] Berkeley Function Calling Paper Club! — Sam Julien, Writer
Assertion Supported
Julien: BFCL v3 shifted from AST to state-based evaluation
“So they really updated their evaluation from the abstract syntax tree to something called state-based evaluation, which we'll, we'll talk about in just a second.”
Sam Julien Oct 5, 2024 ▶ 8:24 [Paper Club] Berkeley Function Calling Paper Club! — Sam Julien, Writer
Assertion Not yet assessed · timeframe Oct 2024
BFCL v3 generates evaluation tasks using graph edge construction
“They basically created their own API for the sake of this testing, and then did this like mapping to create a graph edge construction and like generate tasks through that.”
Sam Julien Oct 5, 2024 ▶ 10:46 [Paper Club] Berkeley Function Calling Paper Club! — Sam Julien, Writer

The other half of the tape: Sam Julien's own voice is left out of every number here. Other people bring the name up 2 times in 1 episode on Latent Space. every mention, with the transcript →

Every mention by year

tap a year for its mentions
0011212024episodesmentions
0112024episodes it came up in
0010.5212024episodesmentions per episode
2024 2 mentions in 1 episode

Appearances (1)

EpisodeDateSpeaking time
[Paper Club] Berkeley Function Calling Paper Club! — Sam Julien, Writer Oct 5, 2024 20m
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more