Cognition AI CEO Scott Wu discusses historical progress metrics in AI agent capabilities with Joe Lonsdale.
Prediction Not checkable as stated
Wu: Reinforcement learning will eventually beat any benchmark with a clear feedback loop
“I think the natural conclusion of RL, which is what we're kind of getting to, is you basically can solve any benchmark, which is insane to think about... Which means like, if you have a clean set of environments, if you have a good feedback loop to decide what…”
Assertion Not checkable as stated
Wu: Devin routinely delivers 6x to 20x efficiency gains for enterprise customers
“We pretty routinely see spots where we can make things like anywhere from six X to 20 X more efficient.”
Prediction Not checkable as stated
Wu: Within five years, software engineering will just be describing what to build
“And I think that's kind of what we're going to get to over the next, I'll call it like five or so years is a point where Software engineering. We might even still call it software engineering or programming or whatever, but it's, it really is just you telling …”
Opinion
Wu: Students should still study computer science despite AI coding advances
“People ask me all the time, you know, it's my son or daughter is, 16, you know, should they even be studying computer science? And I always say, yeah, absolutely. And I think the reason is because these are, you know, the concept level things that actually all…”
Prediction Not checkable as stated
Wu: Lowering barrier to coding will drastically increase total software built
“And I think we're going to see something similar here where it's, you know, imagine, imagine the w w w what we will do, you know, when we get to the point where everyone can just build products and websites and apps and whatever it is of that level of quality …”
Insight
Wu: Raw intelligence is no longer the bottleneck in AI
“I think what it kind of shows, I would say is actually, I think we're at a point now where intelligence actually isn't the bottleneck, if that makes sense. You know, I think these models are capable of solving some really, really hard problems.”