Braintrust, every mention
36 scenes · ← back to Braintrust
tap a year for its mentions
every year anyone Ankur Goyal 40Shawn Wang 6Brendan Fortuna 3Alex Krentsel 3Alessio Fanelli 2Sujay Jayakar 1Samuel Colvin 1Emily Glassberg Sands 1Aaron Levie 1
Verbatim, from the transcripts: the passages where Braintrust comes up
Exo: Harnesses should see their own code and logs — Alex Krentsel, UC Berekeley / Google Research
- ▶ 34:19 Alex Krentsel The EXO harness and agents built over top of it is running in production at Braintrust.
- ▶ 35:02 Alex Krentsel He sees a lot of how people are using agents out in, uh, in his role at BrainTrust and is just also, yeah, he's excellent to work with. 2 times in the scene
- ▶ 41:59 unnamed speaker Well, you know, if you ever need an evals guy, I've heard brain trust is pretty good.
⚡️Monty: the ultrafast Python interpreter by Agents for Agents — Samuel Colvin, Pydantic
- ▶ 1:15 Samuel Colvin So we're weird because we're the only, I guess we mostly consider ourselves against the other AI observability platforms, Braintrust, Langsmith, Langfuse, Arise, those guys, but we're actually full observability, full open telemetry.
Why Every Agent Needs a Box — Aaron Levie, Box
- ▶ 34:45 Aaron Levie I'm super excited for what brain trust is doing, excited for, you know, Langsmith, all the things.
Brex’s AI Hail Mary — With CTO James Reggio (acquired for $5B by Capital One!)
- ▶ 56:00 Alessio Fanelli Yeah, I work with a company called Barisai that does user simulations, and I think like that's what's been interesting.
The Great Evals Debate — Ankur Goyal & Malte Ubl
- ▶ 0:44 Shawn Wang So I wanted to invite Ankur and Malta from Braintrust and Vercel, uh, to just talk about like the state of evals here and like the evals debate.
- ▶ 3:25 Ankur Goyal And I think what I've seen both firsthand from building Loop, which is our own agent and brain trust, but much more secondhand from working with Malta and a bunch of really great companies building AI products is that the best people are…
- ▶ 10:07 Ankur Goyal In fact, when we started BrainTrust and we talked to customers, we're, um, sort of very excited about BrainTrust helping them create golden data sets. 2 times in the scene
- ▶ 14:36 Shawn Wang Okay, so, um, and I wanted to maybe ask Malta, like, are there any best practices that you guys have had developed on eval, maybe around brain trust, or like principles that you, that you think that maybe Ankur hasn't really covered… 2 times in the scene
- ▶ 23:44 Ankur Goyal And I think that what we have really tried to solve for at Braintrust is helping people solve the data problem.
- ▶ 33:41 Ankur Goyal Well, we're shipping crazy stuff every week, so I, I don't, um, I'm not sure I have anything specific to plug, but come visit braintrust.dev and join, join the party. 3 times in the scene
⚡️ Building the AI Hardware Engineer with Matthias Wagner, Co-founder of Flux
- ▶ 36:40 unnamed speaker Braintrust has that.
The Agents Economy Backbone - with Emily Glassberg Sands, Head of Data & AI at Stripe
- ▶ 1:13:53 Emily Glassberg Sands Brain Trust ended up winning.
Why RL Won — Kyle Corbitt, OpenPipe (acq. CoreWeave)
- ▶ 44:46 Shawn Wang So we have an answer from Ankur, from Franchus, where he was like, it's five percent and going down.
⚡️Using RFT to Build Clinical Superintelligence
- ▶ 15:28 Brendan Fortuna One of the cool tools that we use internally is Braintrust. 3 times in the scene
Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
- ▶ 17:58 Sujay Jayakar Oh, I mean, we use brain trust for tracking all this quantitatively, but I can't remember off the top of my head, but it's not like a slam dunk.
Why every AI Engineer needs an AI Gateway (ft Portkey.ai CEO)
- ▶ 21:52 unnamed speaker Human loop, brain trust, whatever, like views.
2024 Year in Review: The Big Scaling Debate, the Four Wars of AI, Top Themes and the Rise of Agents
- ▶ 52:41 Shawn Wang I put the latest chart on the Braintrust episode.
- ▶ 1:45:51 Alessio Fanelli The next one is Braintrust.
Agents @ Work: Lindy.ai (with live demo!)
- ▶ 28:37 unnamed speaker Do you use a off the shelf tool, which is at Braintrust on the podcast, or did you just build your own? 4 times in the scene
Production AI Engineering starts with Evals
- ▶ 0:23 unnamed speaker You ran it for six years, got acquired into Figma, where you were at for eight months, and you just celebrated your one-year anniversary at Braintrust.
- ▶ 12:50 Ankur Goyal We could at Single Store, or now we can at Braintrust.
- ▶ 15:42 Ankur Goyal I would say this in very stark contrast to Braintrust, where if you look at the logos on our website, almost all of the CEOs or CTOs or founders are daily active users of the product themselves, right? 2 times in the scene
- ▶ 18:38 Ankur Goyal And by the way, I know we don't want to talk about brain trust yet, but 2 times in the scene
- ▶ 29:15 unnamed speaker So we're going to go into brain trust now about 20 something minutes into the podcast. 2 times in the scene
- ▶ 36:01 unnamed speaker Well, maybe you should capture a statement of, like, what is Braintrust today? 6 times in the scene
- ▶ 41:53 Ankur Goyal They could be prompts that you save in Braintrust or like prompts that you just type on the fly against 6 times in the scene
- ▶ 54:58 Ankur Goyal And then Braintrust basically like takes that specification of an eval, 6 times in the scene
- ▶ 58:44 Ankur Goyal So when we started, there were a bunch of things that people thought were stupid about brain trust. 6 times in the scene
- ▶ 1:07:02 Ankur Goyal And Zapier was not a customer yet, but they, you know, they were interested in brain trust. 5 times in the scene
- ▶ 1:14:41 Ankur Goyal Um, it's, it's kind of like a lesson that is invariant to brain trust itself.
- ▶ 1:40:08 Ankur Goyal We use linear at Braintrust, and if a customer finds a bug on Slack, we'll, like, click a button and then file a linear ticket, and it auto-generates a title for the ticket.
- ▶ 1:46:05 unnamed speaker What is one thing you would build if you're not working on brain trust? 3 times in the scene
- ▶ 1:55:18 Ankur Goyal If you're building AI software, if you haven't heard of BrainTrust, please check us out. 3 times in the scene
The Ultimate Guide to Prompting - with Sander Schulhoff from LearnPrompting.org
- ▶ 47:00 Shawn Wang I'll, I'll call out a few specialists in this area for, for people to check out a prompt layer, brain trust.