Schluntz: Trust and Auditability Will Be LLM Agents' Biggest Bottleneck
Erik Schluntz · The new Claude 3.5 Sonnet, Computer Use, and Building SOTA Agents — with Erik Schluntz, Anthropic · Nov 28, 2024 · at 1:10:24
Erik Schluntz of Anthropic discusses the key challenges facing autonomous AI agents as they transition to executing multi-hour workflows.
0:00 / 0:26exact quote · 26.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The biggest limiting thing will start to become like, do people trust the output of these agents? And like, how do you trust the output of an agent that did five hours of work for you and is coming back with something? And if you can't find some way to trust that agent's work, it kind of wasn't valuable at all. So I think that's going to be a really important thing is not just doing the work, but doing the work in a trustable, auditable way where you can also explain to the human, hey, here's exactly how this works and why and how I came to it.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Erik Schluntz
Anthropic's Schluntz: Avoid agent frameworks and start from scratch with raw prompts
“I think with agent frameworks in general, they can certainly save you some like boilerplate, but I think there's actually this like downside of making agents too easy, where you end up very quickly, like building a much more complex system than you need. And s…”
Schluntz: Smarter AI models require less agent scaffolding
“And I think like the smarter the models are, the less you need that kind of extra scaffolding.”
Schluntz: JSON Escaping Overhead Degrades LLM Performance Across the Board
“Like if you're trying to output a code in JSON, there's a lot of extra escaping that needs to be done. And that actually hurts model performance across the board. Where versus like if you're in just a single XML tag, there's none of that sort of escaping that …”
Schluntz: AI robotics today is where autonomous driving was 10 years ago
“I think where we are right now is where self-driving cars were 10 years ago. I think we have very cool demos that work. I mean, 10 years ago, you had videos of people driving a car on the highway, driving a car, you know, on a street with a safety driver, but …”
Schluntz: High vehicle costs make Waymo's per-car profitability doubtful
“Those cars are expensive. It's not about if you can hit profitability, it's about your cash conversion cycles. Like is building one Waymo, like how cheap can you make that compared to like how much you're earning sort of as the equivalent of what an Uber drive…”