Claude 3.7 Sonnet post-training heavily biases the model toward CLI tools
Eno Reyes · The AI Coding Factory · May 29, 2025 · at 27:48
Eno Reyes, co-founder of Factory AI, explains how recent frontier model post-training introduces biases toward basic CLI workflows like grep and glob.
0:00 / 0:18exact quote · 18.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So for example, Sonnet 3.7 clearly has it smells like cloud code, right? Same with codex. It very much impacted the way that those models want to write and edit code such that they seem to have a personality that wants to be in a CLI based tool.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Eno Reyes
Enterprises demand full AI delegation, not 15% to 20% speed gains
“I think that the product experience of delegation is really, really immature right now. And most enterprises though, see that as the holy grail, not like going 15% or 20% faster.”
Reyes: Inner-loop coding will soon be fully delegated to AI agents
“The outer loop of software development and what a software developer does, planning, talking with other human beings, interacting around what needs to get done, is something that's going to continue to be very human-driven, while the inner loop, the actual exe…”
Reyes: Developers Should Not Need to Prompt Engineer AI Agents
“A lot of users we believe should not need to prompt engineer agents, right? If your time is being spent hyper optimizing every line and question that you pass to one of these systems, you're going to have a bad time.”
External scaffolding provides higher leverage for coding agents than fine-tuning models
“But our take in general is that freezing the model at a specific quality level and freezing the model at a specific data set just feels like it's lower leverage than continuing to iterate on all these external systems.”
Amplitude and Statsig are closer to semantic AI observability than LLM tools
“I think Amplitude and Statsig and a lot of the feature flag companies actually are closer to this than the existing tools.”
$20/month local IDE pricing limits AI inference quality per task
“The cost when you are local first and your typical consumer is on a free plan or a Like, 20 dollar a month paid plan limits the amount of high quality inference you can do, and the scale or volume of inference you can do per, like, outcome.”