The Ledger, every show

“And now that that has been cracked at this point, I think we actually have a very clear path to just focus on scaling. You know, we were, you know, talking about that you know, the zero to one part, that's not about scaling. I think there's a really strong cas…”

“Sales-led product discovery, you're talking to people from the outside. And again, this is important very early on, but it's not as effective as the FDE-led product discovery, where you're solving these problems from the inside.”

“I think one of the other failures, by the way, that's even prior to that, and more, the easier failure to become a consulting firm, it's where you build the product in the field that the customers are asking for, rather than the one that's actually valuable to…”

“My first, second, and third pieces of advice to people who are thinking about trying an FDU strategy is like, don't, Don't do this at home. If you can avoid it, like it's probably bad for you. Probably you're going to end up doing services. And then only if yo…”

“Probably in five years, we'll look back, we'll be like, well, AI agents, there wasn't even a thing at all, right? We were actually doing all these different things.”

“In the product market fit strategy, you want to be doing less work for every customer, you want to be driving down costs, you want to keep the contract size the same. In the FTE strategy, you want to drive the contract size up. So you're doing more and more va…”

“What the world is gonna look like over the next five years is that the capabilities just race ahead and race ahead and race ahead.”

“And the core idea behind GPT-ONE is that if you have a transformer and you apply this super simple objective of guessing the next token, guessing the next word, that that would be enough signal that you could actually have something that would be able to gener…”

“It is definitely the case that there is a data wall and that if you take the same techniques that we were using to scale LLMs you know, at some point you're going to run into that.”

“Yeah, I would say if you're a founder, the right approach is to start with the very best model you can, because, you know, your startup is only going to be successful if it exploits some, something about AI that realistically is going to be on you know, the fr…”

“I mean, yes, AI has had some effects, you know, particularly on people who write code, but, you know, I don't think you can see it in the productivity statistics, unless it's about how big the data centers are that we're building.”

“I think that the role that we're going to be playing, You know, one, I think there's going to be two roles. One will be something like a lone genius. You know, the Alec Radford of the world working alone at his computer, coming up with some crazy idea, but now…”

“Robotics companies now are where, you know, LLM companies were five years ago. So I think in five years, you know, or even, even sometime in the next five years, we will see the chat GPT moment for robotics.”

“I think weirdly we're going to end up with, you know, automating the scientist, the innovator before we automate, you know, the experiment doer.”

“If you could figure out how to pour enough compute and pour enough data into a neural network, there's really no limit to how, how good it could get.”

“Ah, we think it's actually getting better over time that, you know, some of the reports are sort of confounding various different things.”

“The way I think about it is that we are still pretty far away from the level of scale that is the human brain. And so there's no reason for there to be an asymptote. I think it just keeps working.”

“When you push this to the level where AI is at the human level, I mean, they're basically going to be creating their own civilization.”

“I'm always a little worried about the infrastructure work, because I think, you know, you're solving the problems as they exist today, and when GPD 4.5 and GPD five come out, you know, they're gonna have fundamentally different use cases and fundamentally diff…”

“What he realized we needed was for the FTEs to act as product discovery. So they would go to the site, they would take the product as it was, and they would fill the gap between what the product did and what the users needed. So, you know, the FDE goes and bui…”

“And then for your deltas you want someone who's really good at prototyping. So the wrong profile for a Delta would be someone who's a craftsman who really loves, you know, making sure the abstractions are exactly right. That, you know, they're building softwar…”

“From a business model perspective, one of the key things that you will see, that you should see, is that it may be the case that you're, when you go into, you do a new deployment at a customer, that you're actually losing money early on. As the longer you're a…”

“There's a classic failure mode here where, you know, the FD implements something for one customer and you say, great, well, that's how you should do it. And you bring it directly into the product. And it turns out if you do that, you're building something that…”

“We didn't hire product managers for a long time. And when it did come time for me to hire product managers, I would interview people who were amazing product managers at, you know, other companies. And I would ask them to think at this level of abstraction. Th…”

“Well, that's the advice that you give to an early stage founder, and the FDE model effectively is doing things that don't scale at scale.”

“I think the startups, the most successful startups doing the FTE model have people from Palantir running the FTE model.”

“Early on, it makes sense for the startup to just take on all the risk and say, we're going to just believe in our own execution. And, you know, we're going to take on the risk and you pay us if it works or you pay us when, you know, we're actually able to expa…”

“You want to focus on, on the user experience and you have to do that, but you also have to remember your other key customer is the FDE. Your product should be, you know, ultimately delivering a good thing to the user after FDE customization, but it should be d…”

“And there was real insight that was generated there, which was that it really strengthened our belief that scale was the path to improving artificial intelligence. That with Dota two, the secret idea was that we could take huge amounts of experience and feed i…”

“Early on there were sort of, you know, a couple big projects, as I said, and then some room for exploratory research. And at the very earliest days, the exploratory research was really about you know, it's about what the researcher wanted to do, but also it wa…”

“And so early on the deep mind culture was I, you know, a caricature is Demis had a big plan and he wanted to hire a bunch of researchers so he could tell them to move forward with his plan. And Google brain said, let's rebuild academia. Let's like bring in all…”

“And we took a different approach, which was really more like a startup where there was no sort of big centralized plan. But at the same time, people didn't have, you know, it wasn't just sort of let a thousand flowers bloom. Instead we had opinions about what …”

“I think academia is good for this very narrow thing of, you know, small groups, you know, trying out crazy ideas, but academia has a lot of incentives that prevent people from collaborating. And in particular in academia, there's this obsession with credit.”

“Just getting to that point where it's sort of plausibly begins to work is, is a huge, difficult problem. And it's completely separate from using scaling laws. Now, once you get it to work, that's when scaling laws come into play.”

“What we're going to see out of reasoning, out of long thinking is that it's really going to unlock the possibility of agents to do actions on your behalf, which, you know, has sort of always been possible, but it's just never been quite good enough.”

“There's a rule of thumb that I like, which is basically, you know, if you want to go, if you want to add a nine, if you want to go from 90 to 99% or 99 to 99.9%, That's maybe an order of magnitude increase in compute.”

“I am a little skeptical of the deep emotional connection, you know, that, that, you know, guys are gonna have AI girlfriends. I think that's not what guys are looking for in a girlfriend, frankly.”

“I, at this point, I think we understand neural networks a lot better than we understand the brain. So whenever someone talks to me about the brain, I always think, well, what does a neural network do? And then probably actually the brain does that.”

“Well, that's actually one of the cool things that we've discovered with GPT-III and GPT-IV, Is that the model, so the models are trained on the internet and they're reading everything, right? Then they try to act like what they've seen, right? They're predicti…”

“There's this researcher, Aditya Ramesh. He built a model called DALI, which allowed you to take a description and build an image out of it. He spent two years working on that. Wow. For most of those two years, no one other than him believed it would work.”

“The Dota II team, Jakub Pachoki, who was doing this, had developed a A technique that allowed you to put lots of data and compute into solving the problem. And it was really working for Dota. And he came over and started doing this for robotics. And we went f…”

“I think Dota two is the last one.”

“Robotics are probably going to need AI to write that, to figure out how to build robots for us.”

“Like the difference between neural networks and linear regression was really big. And the difference between neural networks now and neural networks two years ago is just a sequence series of tricks.”

“And so I think a lot of, like near-term safety, a lot of these discussions about bias, like they really are about the fact that the model has learned all of these things, which it's learned a lot of true facts, it's learned a lot of things that are inaccurate,…”

“Well, there's actually thousands of AI trainers behind the scenes who are, you know, we, we're, we give them prompts and they have to score the prompts. And so, you know, I think for better or worse, they bring their biases to it, but we try to make it so the …”

“I think I don't want my kids using it for their homework, but what I want is I want teachers figuring out how to make homework that It requires you to use AI.”

“And again, it was a side project. We called it a low-key research preview. We didn't do any press on it, and it just blew up.”