OpenAI o1

“So it was obviously great to see, like, we still are better than O-one out of the box. You know, even with an older model, and I'm sure that that, that Delta will continue to grow once we're able to train O-one and once we've done more work on our dataset usin…”

“Even now we've started Replacing some of the reasoning traces in our Genie model with reasoning traces generated by O-one, or at least in tandem with O-one, and we've already started seeing improvements in performance from that point.”

“I'm really excited to see things like system prompts, And structured outputs, and function calling, make it into a one, we will be there by the end of the year.”

“Claude Sonnet so far is beating O-one on coding tasks without At least one preview without being a reasoning model and same for Gemini pro or Gemini two point O.”

“What I found that worked really well was not telling the model how to think about it. If that makes any sense, like almost more like you're actually interfacing with a person, which is like scary in its own right. But no, if you're working with a like a cowork…”

“Most people try something once and if it doesn't work, they like They have this like very fixed mental model. They never tried again.”

“It feels like we're, like, going back Into a time where having these separate steps with almost like varying levels of intelligence becomes increasingly important, like this idea of chaining.”

“It's almost like the chat is still the UX, you know, and I think that's what confuses people. Maybe like if you didn't have a chat interface and it was like write a brief, you know, I think people would maybe be steered more the right way.”

“What I do, and I think what you should do is use other prompts to sort of bootstrap your prompts. So like I have it set up in cursor or in windsurf where I was telling Alessio before we started, you basically, I have like a prompt template in a prompts directo…”

“And then we also on this podcast, we interviewed Cosign that actually fine tune four O to, on three bench, sorry to achieve data on three bench. And that score was actually higher than O one when it came out.”

“O-One is very good at programming, but it's kind of, the agent part was the harder part to get it to do here. I think it's like less trained To take the next step in like an agentic task, whereas GPT-IV for like the last two years has been really, you know, pr…”

“It solves, like, something like 57% of problems with a single Rollout and then using parallel rollouts and selecting the best one. With other techniques, we get something like 64%.”

“I tried R one, but R one is a bit under O one with small agents. And I think this is also a matter of formatting. Like sometimes the model struggles to just output them, the code snippets in the correct way that we expect.”

“So I came up with four and the foundational one was skills, which is What I would say that we have already done with O-one and R-one, which is you do a lot of RL, you show the inference time scaling works and you get really high benchmark numbers. And then the…”

“O1's like the initial thinking models were a big deal when I was doing like core academic, like how do I create a performance model for explaining how this kernel behaves? Like, From first principles, that kind of thing was not really in the scope of any model…”