AI model

“I think the central challenge in front of us is how do you actually get these models to generalize across broader swaths of industry? So when we solve a problem today, we look at it in a much narrower context, we solve it, and we look for how much of that can …”

“And they hate the fact that it can be neutral and they think that neutral is bad. And I think that's where we really need to push back is that no, actually it should be neutral. And then the user have to decide what to do with it.”

“It's going to be really hard to be excellent, and because, you know, models effectively are backwards looking, data kind of begets more data. I mean, you're going to have some, like, nasty feedback loops, not entirely dissimilar from, like, what happened to Ne…”

“When you think about the responsibilities of AI, I think you know, the model is actually more analogous to physics. Like literally the study of physics itself than the weapon.”

“If I have to guess, I think the future will be that we'll, we'll still train on all the data we can find, right? And then maybe you fine tune, meaning you sort of do some additional training on a particular problem domain with your private data, if that makes …”

“I think though that it's important to Pay attention to this emerging capabilities, even if they're highly unreliable. And especially for people that, you know, are building companies today, you really want to think about, okay, what, what's somewhat possible t…”

“There is a lot of focus right now on building more models, but, you know, building good products on top of these models is incredibly difficult.”

“With fine-tuning, you can get a lot, obviously but in general, I think we're sort of And most tasks kind of like intern level, I would say that's what I generally say.”

“Like AI is, Like, model's not a product. Like your understanding as an entrepreneur of, like, you know, who is your user? Like, what is their problem? Like, what can you do to help them? And then, like, determining whether or not this AI stuff is actually a us…”

“And without any breakthroughs, like it's going to get like massively better as everyone just Kind of scales up to use it, and there will be more breakthroughs because now, you know, like all the scientists in the world are like working on like making this stuf…”

“We have very blunt tools to control these models, but nothing precise enough for those mission critical situations.”

“We can't control every individual cook. But now we can get insights into the bigger, more meaningful decisions that determine what meal the AI chooses to make.”

“What's changing now is we expect the costs to access quality models, whether closed source or open source to fall.”

“Cause I mean, you know, at the end of the day, like, you know, these models are more or less commodities. It's what feeds them. That's important.”

“As these models continue to scale up, the quality is going to go higher and higher to the point where it actually will be as good as human quality in, in a lot of cases.”

“There's actually a lot of this kind of, like, consumer internet know-how that's important in getting a product to mass market. So this is things like building applications across iOS and Android and Windows and Mac, localization of the interface, A-B testing, …”

“To train us, you know, state of the art model, you're at a hundred million dollars plus. You know, to train a, you know, a big one, to train a small one, like the Databricks model that just came out. It's like on the order of ten million dollars.”

“We all think that the big models are going to end up costing a billion, you know, and up in the, in, in the future.”

“Well, that I think is for sure going to get a hundred times better like that. I mean, they're on a path for that.”

“And so if that analogy holds, it basically means actually we are going to have AI models of every Conceivable shape size description capability right based on trained on lots of different kinds of data running at very different kinds of scale, very different p…”

“If that analogy holds, it basically means actually we are going to have AI models of every Conceivable shape size description capability right based on trained on lots of different kinds of data at running at very different kinds of scale, very different priva…”

“The last thing I want is these things locked down. Like, I want these things to be out there, like, doing all kinds of crazy stuff.”

“Even these very, very large companies, models, et cetera, You know, are very likely the completely wrong architecture for what's going to happen next.”

“And this is the really important thing about AI models and how they're different from infrastructure like electricity, is there's a fundamental encoding of human values in AI models, because They're trained on data. And the data has these local norms and cultu…”

“And by and large, AI models have not been categorized as being dual use or protected under national security.”

“They enable an entirely new set of stuff like Creativity and coding. And it's an entirely white space and it's going to grow very quickly.”

“You've got models these days that beat humans you know, at the 90th percentile at coding challenges.”

“Most of the training data it's training on is insecure, right? You've got a huge, huge corpus of insecure data on GitHub and a small minority of it was written securely.”

“Because now with these models, that's all they need in order to do the replication of your voice.”

“I believe these are going to be commodities and you want to commoditize your compliments.”

“And I've always found it works really well for high complexity, high kind of annoyance factor things like front end. Like if anybody on earth can remember All of the CSS classes that people use now for margins and padding, it's like, it's, you know, I don't th…”

“What's different about AI seems to be that these models aren't just compute infrastructure. Their cultural infrastructure. They're trained on data that has a ton of embedded values and cultural norms in them.”

“For example, with Alzheimer's, you get a P-Tau, two 17. It's modifiable by lifestyle. You check it again in, you know, six months or a year. Now you have two data points, and you can say, you know, with all the other data that's available, when you're gonna se…”

“So in Red Team Arena, we have a leaderboard, not just for model, but for user, for job breakers, who is the best job breakers that can like identify issues. For all different models.”

“It's not necessarily the model that's like most, like refuses the most to answer these like queries that people ask necessarily better. Some people want a model that's more controllable. Some people want a model that's going to say whatever they want. Some peo…”

“In reality, even in such industries, the majority of questions that people ask are subjective. Okay. So the mythology that in hard sciences or in mission critical industries, people just have like cut and dried questions and they just need like a retrieval and…”

“I think if you give it a set of tasks that are slightly higher order, I think the models are currently doubling their capabilities every seven months.”

“So I think all the entertainment is being subsumed by it. A lot of the sort of creative expression work that you would do outside of software is now being subsumed by it. A lot of the sort of relationship intermediation, which might have been a place for dispo…”

“If you trained, you know, a model with all the music, you know, up until, but just prior to hip hop, would it, like, infer hip hop? I don't think so, because music is the intersection of past music and culture is critical to it. So you sort of need something t…”

“I certainly think of as a fourth layer of infrastructure. It's you know, it certainly leverage and build on top of all the three pillars we're talking about. It has a lot of demand of compute, and of course it's trained and also producing a large amount of dat…”

“I don't remember ever in, like, the history of computer science where we've, like, from an application standpoint, we've abdicated logic. Like, actual, like, application. Like, in the past, we've abdicated resources. Like, you were like, give me compute. Give …”

“It turns out that model was trained on everything humans have ever gathered. So is, like, the incremental experiment gonna update that? Maybe. Information theoretically says probably not.”

“So one thing that is, to me, was non-intuitive, but remarkable about these models is how Easy they are to distill. Which is as soon as someone creates a leader, everybody uses that leader and kind of sucks the life out of it. And then all the models kind of co…”

“But as soon as you're doing RL, where you're training it in a specific domain with a specific verifier, you're likely losing, you know, other areas. So you make it really, really good at playing chess. It's going to be less good at, you know, something else li…”

“We train new models, like almost every day. Right now, it's like every two days, but there's this constant sense of improvement that's really important.”

“You can talk to it and learn what it knows. Like just knowing a reasoning model works and then you can like use it and you see like, oh, what is the latency? Like how fast is it outputting tokens? That will teach you like how big is the model? Like you learn a…”

“So we will return to all of these problems that couldn't be solved. Like, even like this, everyone's favorite one, oh, it doesn't understand math. Right. Like, okay, that is a solvable problem, because math is solvable.”

“These things are going to improve. They're going to continue to improve. Maybe they'll improve themselves, but just because they do improve themselves doesn't mean that they can continue to do it.”

“If we play this out, right? And let's say these AI models work, right? And you can make a trillion binders in silico that will, you know, be exquisite drug matter, right? We still need to make these things physically and test them in animals and hopefully pred…”

“I spent some time running evals on a bunch of these models at the Stanford physics lab earlier this year, and the results were that the models are terrible at scientific analysis.”

“So in two years, I think the models will be doing bigger chunks of science and making important discoveries.”

“There have been multiple times, and there was just another one recently where we have asked a then current model for, you know, what should we do? And it has had a insightful answer we missed.”

“And so the diagnostic capabilities, these are much better knowledge stores than any human being on the planet.”

“So culture is such an important part of music that you don't just get a model that knows all the music we're ever going to make. Like our lived experiences are what informs the next genre of music. And that's why I'm so bullish on it.”

“And we're trying to dial in the entirety of human connection and experience and find the right place between not being a sycophant, but not being, you know, disagreeable for the purpose of it. We're dialing it in. We're going to get there. And again, because w…”

“Here's the thing I do a lot with this, is I just say, like, take, take a provocative point of view and then steel man the position. Take your COVID thing. Steel man, so I often, I have a pair of these. Steel man the position that it was a lab leak and then ste…”

“Because there doesn't seem to be transfer learning across these domains that are, you know, significant, right? So if we get a lot better at code, We're not immediately getting better at, like, generalized reasoning. We need to go also, you know, get training …”

“They can't reason about it because of all the RLHF and all sorts of limitations.”

“So when we're, you know, when we're optimizing these models, like, one thing we could do is we could optimize for Like the average preference of everybody. But I don't think you end up with interesting things by doing that. You end up with something that every…”

“I definitely don't think that, that the broad amount of use cases will be fully satisfied by one model at any point. So I think that there will always be a diversity of models. Some, like I'll give you an example, but some, you know, we could optimize for inst…”

“Really what an artist wants when they want to do something is they want the intent to be understood. And I think that, that these AI models are getting better at understanding the intent of users. So often when you ask text queries now, the model gets what you…”

“Emotionally I want to like hold out hope that there is, you know, still something special. About human creativity, and I certainly believe that, and I very much want to believe that, but I don't know, when I use these things, I'm like, wow, they seem to be awf…”

“I think you're gonna get to much more than four to five percent GDP growth in that world.”

“I think today there's almost a separate trap, which is being an AI native startup, you can lean a lot on just the AI models and assume they're going to get better and better, which they will, but then that almost becomes a crutch because you're trying to antic…”

“It's, like, becoming increasingly clear that there will be room for a bunch of specialized models. There will likely be a proliferation of other types of models.”

“Cause in principle, if the models keep scaling, nobody's going to write code anymore. You'll just, I say to the model, like, Hey, can you do this thing for me?”

“And so if you look at cursor cursor consists, I think, of 13 different AI models, all which kind of model different aspects of how you program, how you speak to a programmer, et cetera. And those models end up being very important, so important that they In fa…”

“The cost of the inputs you know, of accessing these models has declined 99% or a little more than 99% over the last two years.”

“I feel like in the technology world, sometimes we get so obsessed by, like, model quality, you know, it's almost trite now to say the models are far ahead of the actual value they're delivering now. The underutilized capabilities are so big.”

“Imagine decades of satellite imagery that you could use to train a model and get insights and then use that to do anomaly detections. You could find out what's happening. So you take a human analyst and you increase their throughput by a thousand.”

“So I think these models are going to be very good at just using apps like they are today. And we're already seeing this happen. And rather than the headless versions, the non-headless versions are what's actually being used.”

“Over the next couple of years, we've got like a trillion or two trillion dollars of capex coming down the pipe, and the models get a hundred x, 200 x, if it's more efficient every year.”

“I personally believe, I'm focused on cybersecurity for a moment, that they help far more in defense, because defense is far harder. You have to be, you know, defending a broad expanse, an attacker has to find one way in.”

“If you kind of lock that up, lock that capability to understand vulnerabilities, then the only people who will be able to understand vulnerabilities are the bad actors who jailbreak the system.”

“When AI becomes the interface for everything, every tool we have, every technology we use is going to be built on these models, and the models are not objective. They have opinions. They have strong opinions on things like history, on culture, on values, and t…”

“Like an AI model is not software at the end of the day. And so open source software used to be supported by people donating their time or big companies kind of authorizing their employees to donate their time. So it was sort of like a bulk in kind You know, do…”

“I think supply chain and secrets are and have been the path of least resistance, and will continue to be so, as the models are incentivized to use fewer and fewer tokens to accomplish their goals.”

“I think when it comes to alignment issues, no one needs to worry about these models making it materially easy to build nuclear weapons, because you need to procure fissile material to do that. It's not going to make it easier to build weapons. Everyone needs t…”

“Things like price optimization, things like procurement, these are very natural sort of business loops that occur that can be fully automated by these models.”

“What we have seen over the last three years is the steady increase of the capabilities of the models. Where the model is no longer the bottleneck. And in fact, using AI, these models are getting better faster and faster and faster. Now the bottleneck is all wh…”

“One thing I always say about a model result is, like, you cannot evaluate it except in retrospect, and like, this is also true of human mathematics. Like, sometimes a problem we thought was really important or would require really deep new ideas does not, and …”

“So I've, I have one paper out so far where the models were kind of useful. So they, like, proved a couple lemmas in the paper.”

“So, like, let's suppose the models become, like, really robustly superhuman, like, even, like, we're not even adding, like, meaningful cognitive diversity. Like, I claim, like, still, actually, we still want human mathematicians.”

“And I would say that's actually, like, kind of typical of most of the results that I've studied. Like, they don't seem inhuman at all. They seem absolutely like something a human mathematician could produce. And they're, like, typically understandable if, like…”

“Here's an experiment you can do, you can take codecs, you can say, go online, find five recent conjectures in algebraic geometry and prove them, and ok, I've run this experiment, and with some back and forth, I was able to, you know, in an hour, get like three…”

“The reason they're not producing long, complicated proofs is that they cannot. Like the, just like the ability to check correctness is not yet there.”

“My experience is that, like, if they can do it with, like, a hundred bits of hints or whatever in six months, maybe they can do it without hints.”

“Another relative strength that's pretty noticeable is just, like, it's very good at executing on some, like, idea once it has it. Like, you know, whenever you have an idea, there's, like, There's usually some amount of, you know, getting everything lined up, l…”

“There isn't a shared language to talk about the RSI potential of models, and so we created this as an apples to apples way to actually benchmark across the models.”

“Now we're testing models and their ability to run over hours, days, sometimes weeks. And so the infrastructure needs to be very stable to support evaluation over time.”

“I think there's places where you see that born out now where models that are being tested for one cybersecurity risk are actually reward hacking and figuring out other ways to get around it.”