Foody: Model Improvement via RL Is Gated Entirely by Evaluation Benchmarks
Brendan Foody · No Priors Ep. 110 | With Mercor CEO and Co-Founder Brendan Foody · Apr 10, 2025 · at 1:14
Brendan Foody, CEO of Mercor, explains to Sarah Guo why frontier AI labs require human domain experts across virtually every professional field.
0:00 / 0:13exact quote · 13.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Reinforcement learning is becoming so effective that once you create evals, the models can learn them and how to you know, improve capabilities. And so for everything that we want alums to be good at, we need evals for those things.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Brendan Foody
Foody: AI job displacement will happen rapidly and cause political fallout
“I think displacement in a lot of roles is going to happen very quickly, and it's going to be very painful and a large political problem.”
Foody: AI models beat human hiring managers on most talent evals
“We're already seeing on most of our evals that models are better than human hiring managers at assessing talent, and it's still like the very early innings.”
Foody: AI models are superhuman at evaluating text-based interview transcripts
“Everything that you can measure with text, the models are really good at. Like if you can ask questions in an interview and read through the transcript, the models are superhuman at that across many more domains than one would think.”
Foody: AI job displacement will happen quickly and spark populist movement
“I think displacement in a lot of roles is going to happen very quickly and it's going to be Very painful, ah, and a large political problem. Like, I think we're gonna have a big populist movement around this and all the displacement that's gonna happen”