The Ledger, every show

“So they're actually opening 70% more PRs and than the engineers who aren't using Codex as much and the gap is widening.”

“My sense is I think managers will be able to manage much larger teams in this world. Kind of like how, you know, like software engineers are managing 20 to 30 codexes. My sense of these tools will allow managers, people manage to be higher leverage and will al…”

“It's like, yeah, if, you know, if people are so high leverage, at some point there will likely be a one person billion dollar startup.”

“You know, it might, we might end up in in a world where there's just, like, a handful of big players that are offering platforms and supporting all of these startups, but, you know, the types of venture-scale return startups that can really hundred or thousand…”

“We actually did a custom on-prem deployment with them onto one of their supercomputers called Venado. And so this actually involves a bunch of, you know very bespoke work with some FDs also with a lot of our developer team. To actually bring one of our reasoni…”

“I think there was an eval that showed that hallucinations basically went to zero for a lot of this.”

“So, 95% of engineers use Codex. 100% of our PRs are reviewed by Codex daily as well, so basically any code that goes into production that's merged in, Codex kind of has its eyes on and suggests improvements, suggests changes in the PRs.”

“And it makes, you know, code reviews go from a, you know, I don't know, 10:15 minute task to sometimes even just like a two to three minute task, because you have a bunch of suggestions already, already baked in.”

“Almost every engineer heavily uses Codex in all of their tasks at this point, and so I, you know, if I were to guesstimate, like, the vast majority of code at this point is, It was probably authored by AI.”

“If you follow this trend, like, I think, like, in the next 12 to 18 months, we could see models that could do multi-hour long tasks very, very coherently. At some point, it might reach, like, you know, six hours a day long task.”

“I think they're gonna get a lot better at audio over the next six to 12 months, especially the likes, you know, the native multimodal models, the speech-to-speech ones.”

“Every single model we've released in one of our products gets released in the API.”

“The other cool thing is it's actually being shared between Los Alamos and some of the other labs Lawrence Livermore Sandia as well because it, it's the supercomputer setup where they can all kind of connect with it remotely.”

“These like unsolved problems that none of the other models could handle, you throw out a GPT-V Pro, and it just like one shots it is pretty crazy, but the trade-off here is you're waiting for 10 minutes.”

“We are in a GPU crunch, so we'll see how, you know, how long that goes.”

“Obviously we were the first one to launch responses API, but like a couple of other people have kind of adopted, I think Grok has it in their API. I think I saw LMSYS just did something”

“We have a really cool setup with Open Router, where we're working with them, and then you can bring your Open Router setup. And then with that, you can actually, you know, you write your evals using our data sets tool, or use our data set tool to create a bunc…”

“If you sign into your ChatGPT account the Siri integration will actually use your subscription status to decide what type of model to use when it passes things over to ChatGPT. And so if you're you know just a free user you get, you know, the free model. But i…”

“What a lot of the interns would do is just, like, full YOLO mode, like, trust it to, like, write the whole feature. And it, like, it doesn't work. It, like, doesn't work sometimes. But, like, I don't know, like, 30, 40% of the time it just, like, one-shots it.”

“We actually zoomed past that.”

“ChatGPT obviously is really, really, really big now. [233] Sherwin Wu: It's I think like the fifth largest website in the world.”

“The biggest product that we have is obviously our developer platform, which is our API. [271] Sherwin Wu: You know, many developers, you know, the majority of the startup ecosystem builds on top of this, as well as a lot of digital natives, Fortune 500 enterpr…”

“The first one was right when I joined the company in September, 20, 22. We, it was pre-TiGPT. But at the time, GPT-IV already existed internally.”

“The most popular one that we have right now is one called Responses API.”

“We actually have a member of our team, Nick Cooper, who is sitting on kind of like that, that steering committee for MCP as well.”

“Right when I joined OpenAI, like, this has actually been, I think, a passion project of John's. Like, he's been talking about doing something in this, like, in this shape for a while, which is, like, a truly, like, low-level research, like, fine-tuning library…”

“When I joined OpenAI around three years ago to work on the API, it was actually the only product that we had. [178] Sherwin Wu: So I think a lot of people actually forget this, where the original product for, from OpenAI actually was not ChatGPT. [183] Sherwin…”

“And a lot of the improvements that we actually got into the model came out of, you know, the learnings that we have from T-Mobile. It brings in a lot of other change from other customers, but because we were so deeply embedded into T-Mobile and we were able to…”

“There's another startup called Accordance that's doing this in the tax space. I think they've been targeting an eval called Tax Bench, which looks at, you know, CPA style tasks as well. And because they, because, you know, they're able to turn it into a very g…”