The Ledger, every show

“And now that that has been cracked at this point, I think we actually have a very clear path to just focus on scaling. You know, we were, you know, talking about that you know, the zero to one part, that's not about scaling. I think there's a really strong cas…”

“Probably in five years, we'll look back, we'll be like, well, AI agents, there wasn't even a thing at all, right? We were actually doing all these different things.”

“What the world is gonna look like over the next five years is that the capabilities just race ahead and race ahead and race ahead.”

“And the core idea behind GPT-ONE is that if you have a transformer and you apply this super simple objective of guessing the next token, guessing the next word, that that would be enough signal that you could actually have something that would be able to gener…”

“It is definitely the case that there is a data wall and that if you take the same techniques that we were using to scale LLMs you know, at some point you're going to run into that.”

“I mean, yes, AI has had some effects, you know, particularly on people who write code, but, you know, I don't think you can see it in the productivity statistics, unless it's about how big the data centers are that we're building.”

“I think that the role that we're going to be playing, You know, one, I think there's going to be two roles. One will be something like a lone genius. You know, the Alec Radford of the world working alone at his computer, coming up with some crazy idea, but now…”

“Robotics companies now are where, you know, LLM companies were five years ago. So I think in five years, you know, or even, even sometime in the next five years, we will see the chat GPT moment for robotics.”

“I think weirdly we're going to end up with, you know, automating the scientist, the innovator before we automate, you know, the experiment doer.”

“Ah, we think it's actually getting better over time that, you know, some of the reports are sort of confounding various different things.”

“The way I think about it is that we are still pretty far away from the level of scale that is the human brain. And so there's no reason for there to be an asymptote. I think it just keeps working.”

“When you push this to the level where AI is at the human level, I mean, they're basically going to be creating their own civilization.”

“I'm always a little worried about the infrastructure work, because I think, you know, you're solving the problems as they exist today, and when GPD 4.5 and GPD five come out, you know, they're gonna have fundamentally different use cases and fundamentally diff…”

“We didn't hire product managers for a long time. And when it did come time for me to hire product managers, I would interview people who were amazing product managers at, you know, other companies. And I would ask them to think at this level of abstraction. Th…”

“Early on there were sort of, you know, a couple big projects, as I said, and then some room for exploratory research. And at the very earliest days, the exploratory research was really about you know, it's about what the researcher wanted to do, but also it wa…”

“What we're going to see out of reasoning, out of long thinking is that it's really going to unlock the possibility of agents to do actions on your behalf, which, you know, has sort of always been possible, but it's just never been quite good enough.”

“There's this researcher, Aditya Ramesh. He built a model called DALI, which allowed you to take a description and build an image out of it. He spent two years working on that. Wow. For most of those two years, no one other than him believed it would work.”

“The Dota II team, Jakub Pachoki, who was doing this, had developed a A technique that allowed you to put lots of data and compute into solving the problem. And it was really working for Dota. And he came over and started doing this for robotics. And we went f…”

“Robotics are probably going to need AI to write that, to figure out how to build robots for us.”

“And again, it was a side project. We called it a low-key research preview. We didn't do any press on it, and it just blew up.”

“You know, the interesting thing about AI is, in the end, it's going to be limited by things like power. You know, if you're, and, you know, chips, and if you're trying to build these really massive clusters in 10 years or whatever.”

“And I think this is where Shamsankar, who's very early employee, you know, now that I think the president and CTO of Palantir, he really invented the FDE strategy.”

“And one of the early robotics papers, we actually said site as open AI because we didn't want to get into a fight. You know, the first author is the one who, you know, gets cited and their name shows up every single time. So we said, you know, we're not going …”

“Yeah, and at some point, there's a Joe Lonsdale neuron that, you know, is out there, you know, and individual people, you can actually find these neurons in the networks.”

“So I think that the thesis behind open AI when I joined in 2017 was that neural networks were the final architecture that could take AI all the way to human level intelligence or AGI.”

“We had already then secretly trained GPT-IV. So this was the first half of last year.”

“The two key roles are those of what we call the echo team and the Delta team. The echo team were Embedded analysts. So they would go to the customer site. They would speak to the users. They would try to figure out what demo or what use case really made sense …”

“I think Aditya Ramesh, who built that model, spent 18 months, maybe two years, just getting to the first version that, that clearly worked.”

“I think it took three years before we had anyone using it to solve.”