Braintrust
also referred to as: brain trust
25 statements across 8 episodes · 19 bullish · 3 bearish · 7 people on the record · first statement Oct 11, 2024 by Ankur Goyal · said 77 times in 15 episodes since 2024 · across every show →
Mentions by year
brought up most by Ankur Goyal (40), Shawn Wang (6), Brendan Fortuna (3), Alex Krentsel (3), Alessio Fanelli (2), Sujay Jayakar (1), Samuel Colvin (1), Emily Glassberg Sands (1)
tap a year for its mentions
2026 7 mentions in 4 episodes 2 per episode
- Exo: Harnesses should see their own code and logs — Alex Krentsel, UC Berekeley / Google Research
- ⚡️Monty: the ultrafast Python interpreter by Agents for Agents — Samuel Colvin, Pydantic
- Why Every Agent Needs a Box — Aaron Levie, Box
- Brex’s AI Hail Mary — With CTO James Reggio (acquired for $5B by Capital One!)
- every mention in 2026, scene by scene →
2025 20 mentions in 8 episodes 3 per episode
- The Great Evals Debate — Ankur Goyal & Malte Ubl
- ⚡️Using RFT to Build Clinical Superintelligence
- 2024 Year in Review: The Big Scaling Debate, the Four Wars of AI, Top Themes and the Rise of Agents
- ⚡️ Building the AI Hardware Engineer with Matthias Wagner, Co-founder of Flux
- The Agents Economy Backbone - with Emily Glassberg Sands, Head of Data & AI at Stripe
- Why RL Won — Kyle Corbitt, OpenPipe (acq. CoreWeave)
- Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
- Why every AI Engineer needs an AI Gateway (ft Portkey.ai CEO)
- every mention in 2025, scene by scene →
Everything said about Braintrust, oldest first
Oct 11, 2024 positive
Oct 11, 2024 bullish
Goyal: Future of AI engineering centers on reusable tools and tight eval loops
“I think it kind of represents the future of AI engineering, one where You can spend a lot of time writing English and sort of crafting the use case itself. You can reuse tools across different use cases. And then most importantly, the development process is ve…”
Oct 11, 2024 positive
Goyal: Matching runtime and eval abstractions eliminates the AI data ETL problem
“If you structure your code so that the same function abstraction that you define to evaluate on equals equals the abstraction that you actually use to run your application, then when you log your application itself, you actually log it in exactly the right for…”
Oct 11, 2024 neutral
Oct 11, 2024 positive
Goyal: Continuous evaluation is the foundational workflow for building superior AI software
“Our core belief is that if you embrace evaluation as The sort of core workflow in AI engineering, meaning every time you make a change, you evaluate it, and you use that to drive the next set of changes that you make, then you're able to build much, much bette…”
Oct 11, 2024 bearish
Oct 11, 2024 bullish
Goyal: Software engineers will drive AI engineering, but ML tools are unusable for them
“The real gap is that software engineers who have a particular way of thinking, a particular set of biases, a particular type of workflow that they run, are going to be the ones who are doing AI engineering, and that the tools that were built for ML are fantast…”
Oct 11, 2024 positive
Oct 11, 2024 positive
Goyal: Adopting evals resolves engineering stalemates over prompt and model choices
“And I think in the absence of evals, what I saw at Impera, and I see with almost all of our Customers before they start using brain trust is this kind of like stalemate between people on which prompt to use or which model to use or which technique to use that …”
Oct 11, 2024 positive
Oct 11, 2024
Oct 11, 2024 bullish
Oct 11, 2024 positive
Goyal: Selling to business units yielded bigger deals than developer sales
“At Impira, I took kind of the popular advice, which is that developers are a terrible market. So we sold to line of business, and there are a number of benefits to that. Like, we were able to sell six- or seven-figure deals much more easily than We could at Si…”
Oct 11, 2024 neutral
Goyal: AI workloads are roughly 25% simple agents and 25% advanced agents
“I'd say like probably 25% of the remaining usage is what you could call like a simple agent. Which is probably, you know, a prompt plus some tools. At least one or perhaps the only tool is a rag type of tool, and it is kind of like an enhanced, you know, chatb…”
Oct 11, 2024 bullish
Oct 11, 2024 bearish
Goyal: Fewer Braintrust customers run fine-tuned models in production than six months ago
“I will say in my own experience with customers as of the recording date today, which is September or something, yeah, very few of our customers are currently fine-tuning models. And I think a very, very small fraction of them are running fine-tuned models in p…”
Oct 11, 2024 positive
Goyal: Simple tool-calling prompts cover 80% to 90% of AI use cases
“For probably 80 or 90% of the use cases that we see with people doing this, like very, very simple, I create a prompt, it calls some tools. I can like very ergonomically write the tools, plug into popular services, et cetera, and then just call them kind of li…”
Nov 15, 2024 positive
Mar 19, 2025 positive
Jul 29, 2025 positive
Ambience Healthcare uses Braintrust for on-premise LLM observability
“One of the cool tools that we use internally is Braintrust. I think they're doing some incredible work over there, building like a tool for domain experts. They do give some built in observability. They let you deploy on premise. It's a fantastic technology. H…”
Oct 30, 2025 positive
Sands: Stripe chose Braintrust for AI evaluations over two dozen vendors
“So we actually had like more than two dozen applicants for this evals RFP. Shawn 'Swyx' Wang: There's no way you can evaluate all of them. Emily Glassberg Sands: Well, we actually, we did. So they wrote like nice one pagers. We read them all. We narrowed it do…”
Dec 7, 2025 negative
Goyal: Creating Golden Datasets for AI Evals Is Wasted Effort
“People don't really want to create golden data sets. It's, I think it's often a wasted effort to the point that you're making. I think the best teams view offline evals as a mechanism of reconciling what they see in production with real users who are using the…”
Dec 7, 2025 positive
Mar 5, 2026 bullish
Levie: Every enterprise will adopt AI agent evaluation and observability tools
“I've been fully convinced that the whole agent observability and eval space is going to be a massive space. I'm super excited for what brain trust is doing, excited for, you know, Langsmith, all the things. And I think what you're going to, I mean, this is lik…”