Fioca: OpenAI evaluates GPT-5 coding models on behavioral software engineering practices
Brian Fioca · ⚡️GPT5-Codex-Max: Training Agents with Personality, Tools & Trust — Brian Fioca + Bill Chen, OpenAI · Dec 26, 2025 · at 4:00
OpenAI engineer Brian Fioca explains the evaluation methodology used during GPT-5 training to assess coding agents beyond raw code completion.
0:00 / 0:11exact quote · 11.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And so these are just best software engineering practices that turn out to be behavior characteristics, and we can measure the model's performance on those behaviors and grade it that way.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Brian Fioca
Fioca: AI models develop operational habits during training analogous to muscle memory
“This is one of the coolest things about, like, model training is literally, like, they develop habits. It's just like a person does. Like, if you're, like, working on some podcasting tool, right, you're really good at editing, and then somebody makes you use a…”
Fioca: Has not hand-written a single line of code in months
“I haven't written a single line of code by hand in months, because I know what I can trust it to do.”
Fioca: Coding agents enable self-customizing software by writing integrations at runtime
“So now if it doesn't have a tool, it can make a tool that it needs to solve a problem, right? So that's like another layer of abstraction and it's not just coding. You can write software that has an agent that can spin up a codex instance and write a custom pl…”
Fioca: Codex Max manages its own context window to run indefinitely
“Codex Max manages its own context window. And so it can run basically forever without you having to worry about it while it's inside of the Codex harness.”
Fioca: Codex Max can run continuously for 24 hours or more
“Max can run for a really long time. We can go 24 hours or more. I've actually, like, sort of had it gone for more than that”
Fioca: GPT-5 matches Codex coding capability but adds step-by-step preambles
“With the five series, because it's more general, and it's just about as good as coding as codex for a lot of things. We've taught it to be more communicative. And so it has preambles before tool calls. It'll say things like, I'm about to go look for this.”