The Ledger, every show

“I'd love to get to the point where you can just ask it how to cure cancer or something, you know. I mean, it's, you know, it seems a few years away for now, but, you know, like.”

“And like, you know, we talked to like some you know, potential product managers early on and they all say the same thing. Oh yeah. Pick your verticals, narrow it down to me, to make it usable. And like, no, we're not going to hire these people. That's like the…”

“But I think that a lot of the core intelligence is going to come from a From these text models.”

“Entertainment is like this two trillion dollar a year industry. And like the dirty secret is that entertainment Is imaginary friends that don't know you exist.”

“Like you want to launch something that's a doctor, It's going to be a lot slower because you want to be really, really, really careful about not providing like false information, but friend, you can do like really fast. Like it's just entertainment. It makes t…”

“If I, like, I want to push this technology ahead fast, like, that's what I want to go with, because like, A, it's, you know, it's ready for an explosion, like right now, not like in five years when we solve all the problems, but like, now.”

“Our goal is to be like an AGI company and a product first company. And the way to do that is by picking the right product that forces us to work on the right things, things that generalize, make the model smarter, make it, like, do what people, you know, what …”

“You know, some people are motivated by publishing, you know, like, I was frustrated I couldn't launch at Google.”

“Yeah, just make the thing smarter. It's gonna have a better theory of mind, and I think that's definitely something massively, massively important. It seems like one of these emergent properties that just Is gonna, going to come with scale.”

“And without any breakthroughs, like it's going to get like massively better as everyone just Kind of scales up to use it, and there will be more breakthroughs because now, you know, like all the scientists in the world are like working on like making this stuf…”

“I don't think anyone's seen like these scaling laws, you know, stop. I think as far as anybody has experimented, stuff just gets, keeps, keeps getting smarter.”

“Yeah, probably the size of the model is the bigger challenge. We can get a lot of data, but actually the number one thing that's important is how much computation you do to train it.”

“We put up as an example, like a psychologist character, like maybe, maybe you want to, you know, talk to something and feel better, you know, and you know, that gets a little bit of use, but then what we hear a lot more from users is like, I'm talking to a vid…”

“We can get a lot of, we can get a lot of data, but like, really, actually the number one thing that's important is how much computation you do to train it.”

“The hardware is progressing so fast that what you could do at a big company, you know, one year, a few years later, you're going to be able to do You know, at the university lab or in your garage.”

“We consider them a feature or at least, okay, basically, our goal, you know, our strategy is, like, launch something general, let people do what they want with it, and if these models are hallucinating, which we, which they certainly are, and we advertise that…”

“And I think a lot of that has to do with the fact that, you know, this field is kind of alchemy. Right now, like no one knows exactly what is going to work.”

“And like, the problem is super simple to define. It's just like predict the next word, the fat cat sat on the, you know, like, okay, well, you know, what comes next? Like it's extremely easy to define. And if you can do a great job of it, like, you know, then …”

“But I think that a lot of the core intelligence is going to come from these text models.”

“So, and at the same time, like, I don't think anyone's seen a wall in terms of how good this stuff is. So I think it will just, it's just gonna keep getting better. I don't know what stops it.”

“I think I'm not that worried about Coming up with data and I feel like we could probably like just generate some more with the AI.”

“Like if you're like such a public, you know, trying to present like one public persona that everyone likes, you're going to end up just being boring essentially. And people just don't want boring. You know, people want like the, you know want to interact with …”

“No. Not yet. Not yet. I think there's a lot of possibility of you know, a lot of potential benefits and yeah, we're going to work on it as the technology improves.”

“The model we're serving now, we, you know, cost us about, like, two million dollars worth of compute cycles to train last year, and could probably repeat it for, like, half a million now.”

“We're going to launch something, tens of IQ points smarter hopefully by the by the end of the year.”

“The more you get to like mission critical, you know, particular use case, the more you get tempted into like writing particular rules and like doing things that will not generalize well.”

“Like operations cost like 10 to the -18 dollars these days”

“The model we're serving now, we trained last summer and spent about two million dollars worth of compute cycles doing it.”

“In the past things seem You know, potentially too much brand risk at larger companies to like actually launch and get it out there.”

“We've gotten testimonials of people who said that they were uncomfortable, you know, talking to other people, and, like, this is great practice. This is actually helped them build up practice in either social situations.”

“As there has been, there will be a big ecosystem of open and closed of, you know, People who aren't sharing their secret sauce with, and people who are sharing all of their secret sauce and the ability to mess around with things at a small scale is going to le…”

“The reason this whole field is working so well is because Now we have this magic hardware that's great at these dense matrix multiplications. And so you can do them like orders of magnitude faster than you can do anything that involves poking around in memory.”

“Really the key insight is that what makes deep learning work is that it is really well suited to modern hardware where, you know, you have the current generation of chips that are great at Matrix multiplies and, you know, other, other forms of things that requ…”

“Yeah, I think, yeah, there's like a ton of great work going on in you know, in trying to figure out what's real and what's hallucinated, of course. I think we'll solve this.”

“Like he started the thing as like a 20% project where like people are encouraged to spend 20% of their time, like doing whatever they want. So and then he just, like, recruited, like, an army of, like, 20% helpers who were, like, ignoring their day jobs and, l…”

“Probably you don't need that high end, like level of intelligence, like To, you know, to do emotion.”

“Like, you know, we don't want to encourage people to, like, harm, you know, hurt themselves or hurt other people or you know, we're blocking porn.”

“An image is worth a thousand words, but it's like a million pixels. So like the text is like a thousand times as dense.”

“Dialogue, like it's world's number one pastime”

“Like, you know, we're going to see like a huge amount of innovation and, you know, what, what's possible in the largest companies now can be possible in, you know, in somebody's academic lab or garage in a few years.”

“If you have a technology that is, like, really, really general and has billions of use cases and, like, ordinary people can use, like, launch it to billions of people.”

“I remember when I joined Google, there were, like, a lot of people working on this enterprise search appliance, which, you know, it was okay. Like, I think maybe somebody had this conventional wisdom that, like B to B is the only way to make money. But, like, …”

“And then it also lets you do all this co-design of like, you get to affect every part of the stack, which is, you know, which is hugely powerful and fun.”

“So rather than directly working on say medical research or something I think I've got a lot more leverage, like let's push AI technology and then, you know, that can help with a lot of the rest of it.”

“I like this, sort of motto of, you know, a billion users inventing a billion use cases, because that, that's sort of the superpower of this technology.”

“And then if you can do it well, then This thing can just talk to you. It can be, you know, the better you do it, the smarter it gets. It's hugely, hugely general and useful, super simple to state.”

“I'd say around 2020 was when sort of, like, really impressive stuff was sort of in the lab, but not launched.”

“So you know, the model we're serving now, we train Last summer and spent about two million dollars worth of compute cycles doing it.”