OpenAI

“Right now it takes, I think, about fifteen million dollars, fourteen million dollars to train these models right now.”

“I'm friends with Sam Altman. He's been asking me for help. I think I disagree with him on like lots of regulation. He wants to put in place, but it goes to the existential stuff.”

“We had already then secretly trained GPT-IV. So this was the first half of last year.”

“There's this researcher, Aditya Ramesh. He built a model called DALI, which allowed you to take a description and build an image out of it. He spent two years working on that. Wow. For most of those two years, no one other than him believed it would work.”

“Well, that's actually one of the cool things that we've discovered with GPT-III and GPT-IV, Is that the model, so the models are trained on the internet and they're reading everything, right? Then they try to act like what they've seen, right? They're predicti…”

“The Dota II team, Jakub Pachoki, who was doing this, had developed a A technique that allowed you to put lots of data and compute into solving the problem. And it was really working for Dota. And he came over and started doing this for robotics. And we went f…”

“I'm always a little worried about the infrastructure work, because I think, you know, you're solving the problems as they exist today, and when GPD 4.5 and GPD five come out, you know, they're gonna have fundamentally different use cases and fundamentally diff…”

“And again, it was a side project. We called it a low-key research preview. We didn't do any press on it, and it just blew up.”

“And so I think a lot of, like near-term safety, a lot of these discussions about bias, like they really are about the fact that the model has learned all of these things, which it's learned a lot of true facts, it's learned a lot of things that are inaccurate,…”

“Ah, we think it's actually getting better over time that, you know, some of the reports are sort of confounding various different things.”

“So I think that the thesis behind open AI when I joined in 2017 was that neural networks were the final architecture that could take AI all the way to human level intelligence or AGI.”

“Well, there's actually thousands of AI trainers behind the scenes who are, you know, we, we're, we give them prompts and they have to score the prompts. And so, you know, I think for better or worse, they bring their biases to it, but we try to make it so the …”

“One of them was a legal tech company. You probably don't even know this. That's partnered with open AI and suddenly sold for like eight times the last round valuation to Thomson Reuters.”

“When you push this to the level where AI is at the human level, I mean, they're basically going to be creating their own civilization.”

“And I feel like at any given moment in time, there's one or two companies that are going through this kind of golden age period where you're expanding rapidly. You're soaking in all the best talent. You're launching multiple product lines. You have this, Diver…”

“Such as the Europeans who are banning OpenAI, by the way, which is very funny for the latest launch.”

“But then you have Apple come out with a paper that shows that if you do minor perturbations, you drop the reasoning score on these systems by like, 65%.”

“And there's a limited number of startups that are going to be able to do the scale compute infrastructure that is required for frontier models, right? Not zero, right? Then we open AI, obviously moving to scale Anthropic, et cetera, but it's going to be a very…”

“OpenAI, I mean, there's already been multiple, multi-billion dollar spin-offs of OpenAI.”

“I think they're bad at enterprise. I think they don't have the, I think they keep trying and they're not succeeding. Maybe they'll get it though. I don't know.”

“We worked with their team for a couple of months, just helping them develop the product and the idea of like what it would even be. And it started from just like chatting with Sam and identifying that there was a gap for a potential social media, new social me…”

“In my year, actually there were a ton of others who all kind of went into AI. And so obviously Steven and Andrew, who started the company you know, who started Cognition with us but also a ton of others. And so Alexander Wang, who started Scale. Demi Guo, who …”

“Anthropic, OpenAI, Google, they're all gonna, in the same way that you probably don't want to build an email company while Google's on its arise, because they're gonna come and eat that, like, every one of these companies is looking at, where are our tokens be…”