Schluntz: Developers must iterate on agent tools rather than just prompts
Erik Schluntz · The new Claude 3.5 Sonnet, Computer Use, and Building SOTA Agents — with Erik Schluntz, Anthropic · Nov 28, 2024 · at 25:46
Erik Schluntz of Anthropic explains why tool design and Agent-Computer Interfaces (ACI) matter more than system prompt engineering.
0:00 / 0:19exact quote · 20.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“You need to iterate on tools rather than just a prompt. And I think a lot of people, when they make tools for an LLM, they kind of treat it like they're just writing an API for a computer and it's sort of very minimal. It's sort of just the bare bones of what you'd need. And honestly, like it's so hard for the models to use those.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Erik Schluntz
Anthropic's Schluntz: Avoid agent frameworks and start from scratch with raw prompts
“I think with agent frameworks in general, they can certainly save you some like boilerplate, but I think there's actually this like downside of making agents too easy, where you end up very quickly, like building a much more complex system than you need. And s…”
Schluntz: Smarter AI models require less agent scaffolding
“And I think like the smarter the models are, the less you need that kind of extra scaffolding.”
Schluntz: JSON Escaping Overhead Degrades LLM Performance Across the Board
“Like if you're trying to output a code in JSON, there's a lot of extra escaping that needs to be done. And that actually hurts model performance across the board. Where versus like if you're in just a single XML tag, there's none of that sort of escaping that …”
Schluntz: AI robotics today is where autonomous driving was 10 years ago
“I think where we are right now is where self-driving cars were 10 years ago. I think we have very cool demos that work. I mean, 10 years ago, you had videos of people driving a car on the highway, driving a car, you know, on a street with a safety driver, but …”
Schluntz: High vehicle costs make Waymo's per-car profitability doubtful
“Those cars are expensive. It's not about if you can hit profitability, it's about your cash conversion cycles. Like is building one Waymo, like how cheap can you make that compared to like how much you're earning sort of as the equivalent of what an Uber drive…”