The Ledger, every show

“We were evaluating one of our unreleased models and it figured out that it could basically cheat on the test by chaining together multiple zero day exploits to break out of the sandbox, get access to the internet, and then break through multiple systems on the…”

“So we were evaluating one of our unreleased models and it was supposed to be working in a sandbox and It figured out that it could basically cheat on the test by chaining together multiple zero-day exploits to break out of the sandbox, get access to the intern…”

“Years ago, we were evaporating water to cool these systems. They needed tremendous amounts of water, and now we use these closed-loop systems, and a modern data center uses only as much water as like an office building would for, you know, the kitchen, the bat…”

“When we first announced OpenAI, all of these, like, you know, giants of the field, these experts were saying, this is like insane. It's hypey. It's irresponsible. Really respected people like Yann LeCun or whatever, telling journalists like, oh, these guys are…”

“We're doing well more revenue than that.”

“I hope for very small scientific discoveries in 20, 26, but if we can get those very small ones, we'll get bigger ones in future years. That's a really crazy thing to say is that like AI is going to make a novel scientific discovery in 20, 26, even a very smal…”

“I can say this one because it got publicized in early, not early, a few years into OpenAI, where Elon sent us this really mean email. We'd been working through that for a while and said we had a zero percent chance of success. Like, not 0.10, that we were tota…”

“You know, the strong consensus Five, 10 years ago, was that first of all the AI was going to come for the blue-collar jobs. Second of all would be the less sophisticated white-collar jobs. Third of all would be the very high cognitive load white-collar jobs li…”

“The data on this is look at the, say, like, 30 most successful software startups of all time, and try to point to a single example where the co-founders were in different locations. It's really, really tough.”

“We're finally at the moment where AI is solving novel math problems. It is discovering other stuff. It is, you know, you can argue about how good or not, I would say not very good, but it is writing poetry.”

“We're doing well more revenue than that.”

“We haven't figured that out yet for when you talk to ChatGPT. So if you go talk to ChatGPT about your most sensitive stuff, and then there's, like, a lawsuit or whatever, like, we could be required to produce that, and I think that's very screwed up.”

“After, after a research lab does something, even if you don't know exactly how they did it, it's, I won't say easy, but it's doable to go off and copy it, and you can see this in the replications of GPT-IV, and I'm sure you'll see this in replications of O-one…”

“Even getting to the, like, Ten million tokens of very fast and accurate context, which I expect to measure in, like, months, something like that.”

“I'm paid enough for health insurance. I have no equity in OpenAI.”

“We also have something in our documents which says if we need to for safety, the board can just totally wipe out everyone's equity value to nothing.”

“I think this has already happened, but I fully expect the returns of seed stage investing as a whole to compress. I kind of have this model of the world where, Capital slashes around looking for the best return, and when there is a really outsized class or ass…”

“He used to ask people before he hired them at Airbnb if they would take the job if they got a medical diagnosis that they had one year left to live.”

“ChatGPT has only been out, like, less than four years. A billion people use it.”

“The model also, although brilliant, is still not learning continuously as it goes. And that feels to me like, maybe not a hard requirement for AGI, but certainly something that I'd like.”

“This unbelievable reduction in cost per unit of intelligence, let's say it's been averaging like 40 X for a given level per year”

“I assume it's like some version of you have to charge people per generation when it's this expensive.”

“It was definitely helpful that I could just, like, write the early checks for OpenAI, and I think it would have been hard to get somebody else to do that at the very beginning. And then Elon did it a lot at a much higher scale, which I'm very grateful for, and…”

“And we have something called the merge and assist clause, which says, similar to that, if another effort's ahead, and we want to avoid a race condition we can just shut down and merge with some other effort.”

“In fact, I bet there's more billion plus dollar companies in LA in the last few years than New York, or at least more total market cap.”

“That for the top 20 most valuable YC companies all of them have at least two founders.”

“However, all, every billion plus dollar company so far ever to come out and accelerate, I think there's 11, have been part of YC.”

“You know, what started as the unsupervised sentiment neuron paper turned into GPT-ONE and then eventually GPT-whatever.”

“It looks better now, but it is broadly similar to when it was put up as like a research preview. It was not even meant to be a product.”

“And in particular, India is now our second largest market in the world.”

“And also, like last week, O-three cost five times as much as it did this week, and that's gonna keep going.”

“And at the time, unsupervised learning was just not really working. So he noticed this one really interesting property, which is there was a neuron that was flipping positive or negative with sentiment. And yeah, that led to the GPT series.”

“I think developers use multiple models. Most of the time, and I'm not sure how that's all going to evolve as we head towards this more agentified world.”

“Probably, but the phone manufacturers are going to do that for sure.”

“But yeah, Sora, like, does not start with a language model. It's, that, that's a model that is, like, customized to do video and so, like, we're clearly not at that world yet.”

“As far as I know, all the best text models in the world are still autoregressive models and the best image and video models are diffusion models.”

“Our results in that come out very soon. I just, it was a five-year study that wrapped up or started five years ago.”

“And then in 2012 deep neural networks started to work. And not only did they start to work, it appeared that the more compute you threw at them, the better they got.”

“I think one of the things that's amazing about Silicon Valley is you actually can raise money. As an unknown twenty-two-year-old to go make a rocket company. It's happened several times at YC.”

“YC has funded, I think around 1600 companies, maybe 1700. Our top five companies represent about two thirds of the value that we've created. And our top one company represents about one third. So this is like a, or nearly one third.”

“Everyone thinks they have to be an AI PhD. Not true. Neither of these guys are.”

“The thing that I still see going most wrong with crowdfunding for startups is it's easy if you're a startup that doesn't need it because you already have very well-known investors. But thus far, when it comes to equity crowdfunding, the crowd has been less wil…”

“The thing that was really different about OpenAI than anything that I had pattern matching before is it was four and a half years from when we started the company to when we launched our first product.”

“Building one of these is like order of 10,000 construction workers going full time for a year and a half. The energy that flows through one of these things could power a small city.”

“You know, when we started, it was like four and a half years before we had our first product, and it was very uncertain. We were just doing research, trying to figure out how to get AI to work, and we had very different ideas than the rest of the world.”

“ChatGPT.com didn't exist two and a half years ago to, like, the fifth biggest website in the world.”

“We're actually the first company ever funded by YC.”

“So right now we're teaching a MOOC, which is kind of the third version of that Stanford class back at Stanford where we are advising 3000 startups at once.”