Ameya Bhatawdekar
Field CTO, Braintrust. On 1 show, 1 appearance. The Shows tab opens the full record on each.
executiveengineerLinkedIn ↗proficient.io ↗
Ameya Bhatawdekar is an AI engineering executive who serves as Field CTO at Braintrust. Previously, he led machine learning engineering for universal search in Dropbox Dash and built large-scale intelligent systems at Microsoft and Facebook.
1shows
1appearances
10statements
1resolved
1supported
0contradicted
100%fully supported
Everything Ameya Bhatawdekar said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Bhatawdekar: Gen AI Systems Require Observability Feedback Loops for Evals
“So when you're building Gen AI systems, you really want that feedback loop of observability that helps you build better evals, that helps you ship better AI.”
Bhatawdekar: AI systems require full reasoning traces to evaluate response quality
“These AI systems need to log the entire trace of how the AI reasoned on the initial input. What were the tool calls it made? How did it interact with the LLMs? How did it sort of ultimately generate the response? And did that response actually meet the user in…”
Bhatawdekar: Span-level scorers pinpoint errors in AI agent execution
“You can define those as deterministic functions, you know, implemented in code, or you can use LLM as judges, but then you can evaluate like, how did each span perform? And that can give you a fairly good way to zero in on problematic areas of your agents.”
Bhatawdekar: Rigorous evals are existential for AI apps built with 'vibe coding'
“When you're building these intelligent agentic applications using Vibe Coding evals almost become existential. You know, that's the only way you have a high degree of confidence that what you've built is going to work well.”
Ameya Bhatawdekar: Enterprise AI quality comes from surrounding engineering, not just models
“These intelligent systems, these AI systems are not just a model, right? There's a lot of layering that happens on top of these models. These systems have to really deliver specific capabilities or specific experiences that help people do certain specific task…”
Bhatawdekar: AI models natively handle orchestration, replacing complex external engineering frameworks
“People built these very fancy frameworks and systems that were fairly complex and complicated to improve the orchestration capability of the model. And, you know, there were some very impressive engineering feats that happened as a result of that. But now the …”
Bhatawdekar: Gen AI Replaced Custom Model Training with Prompt Conditioning
“After the Gen AI revolution, like the way now we build intelligent applications is we take models off the shelf. These are general purpose models. They can reason on a variety of tasks, and then we condition the models to work a specific way by doing prompt en…”
Bhatawdekar: Construction companies use AI to create RFP proposals from drawings
“I've seen systems where construction companies are able to now put together effective proposals using complex engineering drawings, architectural plans, specifications to submit proposals for new RFPs.”
Bhatawdekar: Notion, Stripe, and Zapier run automated evals on every system modification
“All of these companies are building agents, intelligent systems that are performing specialized tasks for whatever use cases they have, and so all of them build evals that reflect how their systems are expected to behave in production. They're all following si…”
Bhatawdekar: Production eval datasets must be continuously updated from live logs
“These eval datasets, they are not static. As the teams look at their logs, at how their systems are working in the real world, in the production use cases, they're able to leverage those insights to continually augment their eval datasets. And so these eval da…”
One line per show, most statements first. The link opens Ameya's full record on that show: the calibration, argument clarity, speaking style and every statement made there.
| Show | Role there | Eps | Statements | Record | |
|---|---|---|---|---|---|
 NEON SHOWLEDGER | Field CTO, Braintrust | 1 | 10 | 100% 1/1 | full record on the Neon Show → |