Krentsel: Self-improving AI requires explicit objective evaluators to avoid misalignment
Alex Krentsel · Exo: Harnesses should see their own code and logs — Alex Krentsel, UC Berekeley / Google Research · Aug 15, 2026 · at 38:37
Alex Krentsel, UC Berkeley researcher and co-creator of Exo, discusses the challenge of goal setting and evaluation in recursive self-improving AI agents.
0:00 / 0:17exact quote · 17.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“In all these optimization problems, the final difficulty is You always have to provide some evaluator and that evaluator has to kind of reflect your goals, because otherwise there's no signal for the thing to hill climb or to optimize, or that signal, it might find a signal for itself, but it might not align with what you actually want.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Alex Krentsel
Krentsel: AI self-improvement requires collapsing outer optimization loops
“You have some outer system that's optimizing some inner system. What if you want to optimize the way you're doing? You're optimizing, then you need some outer, outer loop, and it's this infinite recursion out. And the only way I think out of that is to collaps…”
Krentsel: AI takeoff flywheel will emerge from agents iterating on harness code
“There's a moment like that happening now, and this is the moment that's going to be a flywheel, I feel like, because this takeoff moment will come from being iterating in the same layer that you are producing, I think.”
Krentsel: OpenClaw, Pi, and Claude Code hardcode static agent policies
“These are all policy decisions that are static, that are defined for OpenClaw, or for Pi, or for Clawed code, if you look at their Source code. And so that is the kind of, that is the policy of what an agent is, the tools it can use, the skills it has, how it …”
Krentsel: Improving AI models should architect their own agent harnesses over humans
“We believe All of that needs to be improvable by the agent, especially as the agents keep getting better, because as they keep getting better, it's this better lesson. You don't want to over-specialize because you don't want the human kind of deciding all thes…”
Krentsel: Exo autonomously modified its code to inspect Pokémon game RAM
“We've had XO running, playing, playing Pokemon. And while it's running, the system itself decided to try inspecting the like RAM of the game and then went and mapped the RAM to, and people have reversed in the past, people have reverse engineered this manually…”
Krentsel: Harness architecture must enforce agent rules instead of model alignment
“My bet is this though, that people keep trying to get models to do things that align with their goals and alignment is an unsolved problem. We keep trying to like RLHF, like try to align these models to do the right thing. In harness space, we actually have an…”