Godement: Developers will rely on continuous, automated fine-tuning within years
Olivier Godement · Building AGI in Real Time (OpenAI Dev Day 2024) · Oct 4, 2024 · at 26:46
Olivier Godement (Head of Product for OpenAI Platform) outlines the future product direction for LLM fine-tuning and evals.
0:00 / 0:18exact quote · 18.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The vision we have is, fast forward a couple of years, I think, like, most developers will essentially, like, have an automated, continuous, fine-tuned model. The more, like, you use the model, the more data you pass to the mobile provider, like, the model is automatically, like, fine-tuned, evaluated to get some evil sets.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Olivier Godement
Godement: OpenAI plans API knobs for developers to adjust safety thresholds
“And so I think the direction where we'll go here is that, basically, there will always be, like, you know, a set of behavior that will, you know, just, like, forbid, frankly, because they're illegal against our terms of services, but then there will be, like, …”
Godement: Vision fine-tuning shows higher performance uplift than text
“We've been alpha testing, like, the vision fine tuning, like, for several weeks at that point. We are seeing, like, even higher performance uplift compared to text fine tuning.”
Godement: OpenAI will only build tools closest to the model
“There is no freaking way that OpenAI can build everything. Like, there is just too much to build, frankly. And so, my philosophy is, essentially, we'll focus on, like, the tools which are, like, the closest to the model itself. So that's why you see us, like, …”
Godement: Founders should build apps slightly too difficult for current models
“As a developer, as a founder, you basically want to build an app, which is a bit too difficult for the model today, right? Like what you think is right. It's like sort of working, sometimes not working. And that way, you know, that basically gives us like a go…”