The Exchanges

Q Now we're going to discuss some pretty meaty topics. Before we do dive in, can you just give me the 62nd context on why you're so well versed to discuss them and your roles today?

A So I, I seem to have, be collecting jobs here. I have a number of different roles, um, but I am, I'm first and foremost a professor and the head of the machine learning department at Carnegie Mellon. Uh, I've been here for about 12 years, and here the machine learning department is really kind of unique because it's a whole department just for machine learning, and I've been heading it up actually as of quite recently, kind of get to immerse myself in the business and, and, uh, the thought of machine learning all day, every day. Also, I am, uh, recently on the board of OpenAI, uh, which I joined, uh, at this point a couple weeks ago, and it's been extremely exciting as well.

AI assessment note: “first and foremost a professor and the head of the machine learning department at Carnegie Mellon”

Q If we just took it to a logical extreme though and said we had plundered the reserves of data, And you mentioned that even if we had, we were not seeing a plateauing in performance of models. Why is that? Because one would assume so.

A There are a few different, different sort of notions here. One is just the fact that we are still seem to be in a world where you can increase model size and get better performance, even with the same data. Um, so obviously the real value of bigger models is they can suck up more data, right? They're able to ingest more and more data. But it is also true that if you just take a fixed data set and run over it multiple times, if you use a bigger model, it will often work better, right? So I think that we have not really, we have not really, uh, reached the plateau there. The other thing though, and I think this is, this is a, this is maybe related to your point on synthetic data, and these, these ideas are in the air. We don't, we don't really know the right way of doing this right now. But what I would say is, I don't think Anyone would argue, or most people would, would not argue, that the current models in some sense extract the maximum information possible out of the data that is presented to them. And a very simple example of this is if you train a classifier, uh, just, you know, to, to, to classify images of cats versus dogs on a bunch of images, you get a certain level of performance. If you train a generative model on those exact same images, Generate more synthetic data from that generative model and then train on that more synthetic data. You don't do that much better, …

AI assessment note: “our current algorithms, we are not yet maximally extracting the information from data”

Q And people are saying now, you know, ah, we've got to the stage of diminishing returns, more compute isn't leading to an aligned level of performance in models, um, we've really reached this kind of diminishing returns bottleneck. To what extent is that true, or do we have a lot more room to run in throwing compute?

A I'm not I'm not really sure what the rationale is for saying that we've, we plateaued in the compute sense. Um, most scaling laws that I've seen certainly suggest it can keep going. It's more expensive. You could argue that it's by far just scaling is, may not be the most efficient way to achieve better results. And I actually think that's very likely true. There are other better ways you could argue to Achieve the same level of improvement than compute, but compute still does seem to be both A, a major factor, and B, still seems to improve things. So I'm, I'm, I'm not quite sure. It's, it's more a calculus about also, you know, the, the, the monetary trade-offs of how much models will, will cost at inference time and how much they cost a train, all this kind of stuff. These are much more, I would say, kind of becoming more practical concerns than a concern about the actual limits of scaling.

AI assessment note: “most scaling laws that I've seen certainly suggest it can keep going”

Q What do you think are the other biggest misconceptions that people have towards AGI? There's so many, I mean, I don't think people actually know, but what are others which really frustrate you?

A The thing that frustrates me most, honestly speaking, is the degree of certainty that some people have about whether we will definitely get there very, very soon or even more on the flip side, that there's absolutely no way that we will ever achieve AGI with these current models because of X, Y, Z, right? This, this, this does actually kind of, kind of start to start to irk me a little bit because I don't, I personally, you know, even as a, a product of the AI winter skepticism, I see what's happening in these models, and I am amazed by it. And the people that have been sort of ringing this bell for a while saying, look, this is coming, this is this, you know, they've, in many cases, in my view, been proven right. And I've updated my sort of posterior beliefs based upon the evidence I've seen. And so what irks me the most about a lot of people's sort of philosophy of AGI is that, to a certain extent, how little It seems like observable evidence has changed their beliefs one iota. You know, they had certain beliefs about what it would take to get to general AI, or maybe the AI was, was impossible by definition, or AGI was impossible by definition, and they kind of maintained those beliefs, in my view, uh, in the face of overwhelming evidence, at least pointing to, uh, contrary outcomes.

AI assessment note: “The thing that frustrates me most, honestly speaking, is the degree of certainty”

Q What is the most common question you're asked that you don't think you should be asked?

A Um, the most common one has to do, I would say, with things like putting an overemphasis on the architect, questions that put an overemphasis on the architectures involved in, in AI systems. So, you know, this notion that somehow the transformer was the thing that, that, uh, that, that made all AI possible. I'm often asked questions like, you know, what comes after the transformer and things like that. And, and the reality is, as I said before, and it was probably, I know makes for a good soundbite, Architect, we are arguably in a post architecture phase. They don't, they don't really matter. We could do what we're currently doing with a whole lot of architectures right now, and I hope that I can steer the conversation to more of one where we consider these models not in terms of their particular structure because it's somewhat irrelevant when it comes to capabilities, and we think about these models more in terms of the data that goes into them and the capabilities they produce downstream.

AI assessment note: “I'm often asked questions like, you know, what comes after the transformer”

Q Now I want to start with some foundations and mechanics. When we look at kind of the basic techniques that underpin current AI systems, can you help me understand what are the basic techniques today behind current AI systems?

A Let's talk about AI as LLMs, but with, of course, the context that AI is a much, much broader topic than this. Um, LLMs are, are amazing. Um, the way they work at the most basic level is that you take a lot of data from the internet You train a model, and I know that's a very sort of colloquial term that we use here, but basically what you do is you build a great big set of kind of mathematical equations that will learn to predict the words in the sequence that is given to them. So, you know, if you see the quick brown fox as your starting phrase of a sentence, it will predict the word jumped. Uh, this is a common phrase we use to, I think, use every letter in the English language, uh, in a single phrase. People often use that, and that's what it does. To be clear, we train a big model on predicting words on the internet, um, and then when it comes time to actually speak with an AI system, all we do is we use that model to predict what's the next word in a response. This is, to put it bluntly, A little bit absurd that this works. And I think people often, often sort of, so there's sort of two, two chains, two, two philosophies of thought here. People often use this sort of mechanism of how these models work as, as a way to dismiss them. Oftentimes I know people say, oh, well, AI is, it's just predicting words. That's all it's doing. Therefore it can't be intelligent. It can't b…

AI assessment note: “we train a big model on predicting words on the internet”

Q that, like data, not the bottleneck. Good. Uh, what models? Honestly, again, the joys of this show is I can kind of just regurgitate statements that other smart people have said and test them. Everyone talks to me about kind of moving to this world of many smaller models, which are maybe more efficient. To what extent do we agree with that? Is that right? How should we interpret that?

A I sort of don't really know here, um, to be honest. I think we have not yet, we have not yet reached an equilibrium point where we have a good sense of sort of what the steady state of model size and for what application and how it's being used and, you know, is it being used as a general purpose system or for a very specific reason? Um, this is all still being figured out right now. What I will say Is that I use these models very regularly for my daily work. I work almost exclusively with the largest models that are available to me because it just works better, and when I don't have a given task that I'm doing over and over, when I want to have that generality, I want to work with the larger models that are available. The notion of sort of small language models and this kind of stuff, and again, I, I think this might be Very much a possibility in the future. It kind of comes after the, after we reach this point of generality, right? So once we've done something enough and we realize, okay, there is still a small task we want to do many, many times. Maybe before we would have used custom-trained machine learning model for this, but the idea is that once you have a task, a rote task that you're repeating again and again enough times, And you know a small model can do it. It probably does become valuable to specialize a small model for that task only. But to be honest, I think th…

AI assessment note: “we have not yet reached an equilibrium point where we have a good sense”

Q have already. We will need synthetic data to really supplement the data that we already have, or we need to create new forms, be it the transcription of YouTube videos, which is like a hundred and fifty billion hours or whatever that is. To what extent do you think it's true that we've plundered the data resources that we have available and we are running into a data shortage crisis?

A I think there's sort of two, two kind of answers to this question, uh, which are diametrically opposed. So as, as with many, as with many questions, right? Um, because you're exactly right. You know, the thought is because these models are built to basically predict text on, on the internet, um, if you, if you run out of text, then they kind of, that would imply that they're kind of plateauing, right? That would imply that they're sort of reaching a limit. Um, I don't think this is actually true for, for several reasons, which I can get into, but just from a raw standpoint of training these models, I think there's, there's two ways in which this is sort of maybe true, maybe false. Um, it is true that a lot of the easily available data, sort of the mode, the highest quality data that's out there on the internet has, has, has been consumed by these models, right? We have, we have used this data. There is not another Wikipedia and things like this, right? There's, there's only so much Really high quality, good text that's available out there. On the flip side, and this is the point I often make, um, first of all, we're only talking about text there. We're only talking about publicly available text. If you start talking about internally available text, stuff like this, from a very straightforward standpoint, we have not gotten close to using all the data that's available. Public mo…

AI assessment note: “we have not gotten close to using all the data that's available”

Q What far-flung scenario do you think is most good to think about? Because most people just go, robots, killing jobs, killing humans, ultimately post-killing our jobs.

A I think killing jobs is much more, much more immediate of a concern than killing humans. An example I often use here. To kind of try to bring a little bit of these two sides, the, the, the, the sort of AI taking over the world, uh, killing us all, and kind of the, the more skeptical minded academic folks, we'll say. I see a path. Right now to a world in which, you know, in a few years from now, we start integrating AI models into more and more of our software. We start building it up more and more. We sort of make these things a little bit more autonomous in their actions. We start just naturally because software does everything for us. We start naturally kind of infusing this into all software we have, including software that handles things like critical infrastructures, stuff that controls the power grid, things like this, right? And now all of a sudden you have these agents that are sort of, you know, taking an active Playing an active role in doing things like controlling power grids. This leads to the possibility of, even in my view, sort of massive correlated failures that could do things like bring down power, electricity, in a way that we can't restore it easily in, for a large portion of the country. Now in this world, um, and I, I think it's honestly not, again, if we go down the wrong path, this is definitely not that impossible to imagine. Now in this world where th…

AI assessment note: “massive correlated failures that could do things like bring down power, electricity”

Q God, what, what a breakthrough snap that is. Uh, what did you believe about data that you later changed your mind on?

A Um, kind of on the contrary, I thought that data was sort of, you know, data had to be highly curated to be valuable, and the value in data came essentially from very manual labeling of this data and, and human intensive curation. The big amazing insight Of current AI is that we can, to a large extent, just suck up data that exists out there on the internet, uh, train models based upon that, and get amazing things to come out of it. Not to say there's no value in curation. Of course, there's elements of this, but to a very large extent, this is kind of on the old paradigm of unsupervised learning, and, and that's, that's absolutely incredible.

AI assessment note: “I thought that data was sort of, you know, data had to be highly curated”

Q have already. We will need synthetic data to really supplement the data that we already have, or we need to create new forms, be it the transcription of YouTube videos, which is like a hundred and fifty billion hours or whatever that is. To what extent do you think it's true that we've plundered the data resources that we have available and we are running into a data shortage crisis?

A I think there's sort of two, two kind of answers to this question, uh, which are diametrically opposed. So as, as with many, as with many questions, right? Um, because you're exactly right. You know, the thought is because these models are built to basically predict text on, on the internet, um, if you, if you run out of text, then they kind of, that would imply that they're kind of plateauing, right? That would imply that they're sort of reaching a limit. Um, I don't think this is actually true for, for several reasons, which I can get into, but just from a raw standpoint of training these models, I think there's, there's two ways in which this is sort of maybe true, maybe false. Um, it is true that a lot of the easily available data, sort of the mode, the highest quality data that's out there on the internet has, has, has been consumed by these models, right? We have, we have used this data. There is not another Wikipedia and things like this, right? There's, there's only so much Really high quality, good text that's available out there. On the flip side, and this is the point I often make, um, first of all, we're only talking about text there. We're only talking about publicly available text. If you start talking about internally available text, stuff like this, from a very straightforward standpoint, we have not gotten close to using all the data that's available. Public mo…

AI assessment note: “we have not gotten close to using all the data that's available.”

Q that, like data, not the bottleneck. Good. Uh, what models? Honestly, again, the joys of this show is I can kind of just regurgitate statements that other smart people have said and test them. Everyone talks to me about kind of moving to this world of many smaller models, which are maybe more efficient. To what extent do we agree with that? Is that right? How should we interpret that?

A I sort of don't really know here, um, to be honest. I think we have not yet, we have not yet reached an equilibrium point where we have a good sense of sort of what the steady state of model size and for what application and how it's being used and, you know, is it being used as a general purpose system or for a very specific reason? Um, this is all still being figured out right now. What I will say Is that I use these models very regularly for my daily work. I work almost exclusively with the largest models that are available to me because it just works better, and when I don't have a given task that I'm doing over and over, when I want to have that generality, I want to work with the larger models that are available. The notion of sort of small language models and this kind of stuff, and again, I, I think this might be Very much a possibility in the future. It kind of comes after the, after we reach this point of generality, right? So once we've done something enough and we realize, okay, there is still a small task we want to do many, many times. Maybe before we would have used custom-trained machine learning model for this, but the idea is that once you have a task, a rote task that you're repeating again and again enough times, And you know a small model can do it. It probably does become valuable to specialize a small model for that task only. But to be honest, I think th…

AI assessment note: “we have not yet reached an equilibrium point where we have a good sense”

Q What do you think are the other biggest misconceptions that people have towards AGI? There's so many, I mean, I don't think people actually know, but what are others which really frustrate you?

A The thing that frustrates me most, honestly speaking, is the degree of certainty that some people have about whether we will definitely get there very, very soon or even more on the flip side, that there's absolutely no way that we will ever achieve AGI with these current models because of X, Y, Z, right? This, this, this does actually kind of, kind of start to start to irk me a little bit because I don't, I personally, you know, even as a, a product of the AI winter skepticism, I see what's happening in these models, and I am amazed by it. And the people that have been sort of ringing this bell for a while saying, look, this is coming, this is this, you know, they've, in many cases, in my view, been proven right. And I've updated my sort of posterior beliefs based upon the evidence I've seen. And so what irks me the most about a lot of people's sort of philosophy of AGI is that, to a certain extent, how little It seems like observable evidence has changed their beliefs one iota. You know, they had certain beliefs about what it would take to get to general AI, or maybe the AI was, was impossible by definition, or AGI was impossible by definition, and they kind of maintained those beliefs, in my view, uh, in the face of overwhelming evidence, at least pointing to, uh, contrary outcomes.

AI assessment note: “The thing that frustrates me most, honestly speaking, is the degree of certainty”

Q God, what, what a breakthrough snap that is. Uh, what did you believe about data that you later changed your mind on?

A Um, kind of on the contrary, I thought that data was sort of, you know, data had to be highly curated to be valuable, and the value in data came essentially from very manual labeling of this data and, and human intensive curation. The big amazing insight Of current AI is that we can, to a large extent, just suck up data that exists out there on the internet, uh, train models based upon that, and get amazing things to come out of it. Not to say there's no value in curation. Of course, there's elements of this, but to a very large extent, this is kind of on the old paradigm of unsupervised learning, and, and that's, that's absolutely incredible.

AI assessment note: “I thought that data was sort of, you know, data had to be highly curated”

Q If we just took it to a logical extreme though and said we had plundered the reserves of data, And you mentioned that even if we had, we were not seeing a plateauing in performance of models. Why is that? Because one would assume so.

A There are a few different, different sort of notions here. One is just the fact that we are still seem to be in a world where you can increase model size and get better performance, even with the same data. Um, so obviously the real value of bigger models is they can suck up more data, right? They're able to ingest more and more data. But it is also true that if you just take a fixed data set and run over it multiple times, if you use a bigger model, it will often work better, right? So I think that we have not really, we have not really, uh, reached the plateau there. The other thing though, and I think this is, this is a, this is maybe related to your point on synthetic data, and these, these ideas are in the air. We don't, we don't really know the right way of doing this right now. But what I would say is, I don't think Anyone would argue, or most people would, would not argue, that the current models in some sense extract the maximum information possible out of the data that is presented to them. And a very simple example of this is if you train a classifier, uh, just, you know, to, to, to classify images of cats versus dogs on a bunch of images, you get a certain level of performance. If you train a generative model on those exact same images, Generate more synthetic data from that generative model and then train on that more synthetic data. You don't do that much better, …

AI assessment note: “you can increase model size and get better performance, even with the same data”

Q And people are saying now, you know, ah, we've got to the stage of diminishing returns, more compute isn't leading to an aligned level of performance in models, um, we've really reached this kind of diminishing returns bottleneck. To what extent is that true, or do we have a lot more room to run in throwing compute?

A I'm not I'm not really sure what the rationale is for saying that we've, we plateaued in the compute sense. Um, most scaling laws that I've seen certainly suggest it can keep going. It's more expensive. You could argue that it's by far just scaling is, may not be the most efficient way to achieve better results. And I actually think that's very likely true. There are other better ways you could argue to Achieve the same level of improvement than compute, but compute still does seem to be both A, a major factor, and B, still seems to improve things. So I'm, I'm, I'm not quite sure. It's, it's more a calculus about also, you know, the, the, the monetary trade-offs of how much models will, will cost at inference time and how much they cost a train, all this kind of stuff. These are much more, I would say, kind of becoming more practical concerns than a concern about the actual limits of scaling.

AI assessment note: “most scaling laws that I've seen certainly suggest it can keep going”

Q This is not some- Is that not the same with all new technology introductions to society? There is a gradual curve, and there is, you know, employment displacement, there is societal upheaval, and that is a natural cycle with technology.

A I actually also agree with you, uh, that we will adapt to it. I don't want to Downplay the extent of transformation that might be necessary here. But I also think that the winners in this new world Will the companies that say survive and thrive and become dominant in this new world? Not talking about the AI companies for now, talking about sort of the rest of the companies that the ones that people worry that they're going to fire all their workers because, you know, they can have an AI replace them. The ones that succeed best will not be the ones that fire all their workers to have an AI that does the exact same thing as their old workers. They'll be the ones that understand, okay, what's changing and what are the things that people Can best do now in terms of steering these systems in terms of sort of providing the overall guidance and framework about where we want to go with our, with all this intelligence. The companies that survive, I think will be the ones that best leverage their workforce to achieve To make the best use of this new technology.

AI assessment note: “I actually also agree with you, uh, that we will adapt to it.”

Q Does that not lead to a reduction in the advancement of human knowledge, though, if we only trust the, the people around us? We've known for years when we see them in person, not even when they send us something.

A Yeah, I mean, so obviously there are, there are, uh, you know, massive negative externalities, but we did evolve knowledge at a time, very well at a time before video, right? Before videos existed, we still made scientific progress. So there will be groups that decide that the certain bodies of scientific knowledge are, are valuable, and they will advance those, um, kind of even in light of, Large other portions of the population, which have existed throughout all of history too, that kind of don't value those scientific advances or think differently about the nature of scientific advances. We are already in this world, right? This is the world we already live in. I think it is definitely an accelerant and a, and a shame that this sort of puts us more toward the camp of failing to have an objective reality, but humans And it's arguably our natural state is to not agree on the nature of objective reality. This is a very, I mean, this sounds very negative. I don't want to come across as too negative about this because I, I think that we will still absolutely make progress.

AI assessment note: “I think that we will still absolutely make progress.”

Q This is not some- Is that not the same with all new technology introductions to society? There is a gradual curve, and there is, you know, employment displacement, there is societal upheaval, and that is a natural cycle with technology.

A I actually also agree with you, uh, that we will adapt to it. I don't want to Downplay the extent of transformation that might be necessary here. But I also think that the winners in this new world Will the companies that say survive and thrive and become dominant in this new world? Not talking about the AI companies for now, talking about sort of the rest of the companies that the ones that people worry that they're going to fire all their workers because, you know, they can have an AI replace them. The ones that succeed best will not be the ones that fire all their workers to have an AI that does the exact same thing as their old workers. They'll be the ones that understand, okay, what's changing and what are the things that people Can best do now in terms of steering these systems in terms of sort of providing the overall guidance and framework about where we want to go with our, with all this intelligence. The companies that survive, I think will be the ones that best leverage their workforce to achieve To make the best use of this new technology.

AI assessment note: “I actually also agree with you, uh, that we will adapt to it.”

Q Is that anything I have not asked on AI safety that I should have asked?

A Or one of the biggest questions I am asked about AI safety, because what I laid out to you so far was honestly a, a pretty practical set of recommendations and a pretty pragmatic view on the field. Right? Um, I mean, I'm talking about, you know, preventing jailbreaks. I'm talking about making progress when it comes to securing these models. I'm talking about the interplay between the current release of open source models and closed source models. I do think that While it is not my area, the more far-fetched scenarios about sort of agentic AGI systems that, that start sort of intentionally acting harmful against humans. This is often these, these, so, so I'm thinking about, you know, the, the, the, the rogue AI that decides it wants to wipe out humanity and goes about planning on, on, on how to do this. What seemed to me, and I'll be honest here, far-flung kind of sci-fi-ish scenarios here, These are often the debates we have when it comes to AI safety, and I want to say sort of two things about this. The first is that I think the vast majority of AI safety should not be about these topics. The vast majority should be about quite practical concerns we have on making systems safer, like the kind that I've talked with you about so far. There are already massive safety considerations and risks that are present In current systems and would certainly be present even in slightly more …

AI assessment note: “one of the biggest questions I am asked about AI safety”

Q Does that not lead to a reduction in the advancement of human knowledge, though, if we only trust the, the people around us? We've known for years when we see them in person, not even when they send us something.

A Yeah, I mean, so obviously there are, there are, uh, you know, massive negative externalities, but we did evolve knowledge at a time, very well at a time before video, right? Before videos existed, we still made scientific progress. So there will be groups that decide that the certain bodies of scientific knowledge are, are valuable, and they will advance those, um, kind of even in light of, Large other portions of the population, which have existed throughout all of history too, that kind of don't value those scientific advances or think differently about the nature of scientific advances. We are already in this world, right? This is the world we already live in. I think it is definitely an accelerant and a, and a shame that this sort of puts us more toward the camp of failing to have an objective reality, but humans And it's arguably our natural state is to not agree on the nature of objective reality. This is a very, I mean, this sounds very negative. I don't want to come across as too negative about this because I, I think that we will still absolutely make progress.

AI assessment note: “Before videos existed, we still made scientific progress. So there will be groups”

Q When we think about optimizing the data that we have in terms of kind of value extraction, what could be done further to get further value from the data that we have?

A I don't really know, to be honest. I think this is a major open question right now in research. How do we extract the maximal information content? From these models, or sorry, from the data that we have. Um, but again, as I said, I, I don't think we're close to even extracting all the data that's available, so there's, when I look at this landscape, right, and we know that we aren't close to extracting the maximal information in sort of, you know, the closure set of all the data that we have available, and we aren't come, we have not come close to processing all the data that's available to us. The idea that somehow this is a, a recipe for models plateauing in performance Just doesn't jive to me with the reality of what we see.

AI assessment note: “I don't really know, to be honest. I think this is a major open question”

Q kind of, model ecosystems is just the commoditization of models. I remember, you know, a year ago, 18 months ago, it was so expensive, so hard, there were so few players. Now that there's so many, the commoditization, the reduction in cost, how do you expect this model landscape to play out, given the commoditization being really one of the fastest commoditizing technologies we seem to have seen in years?

A I think that it's been evolving so quickly between Uh, recent releases of open source models, uh, continued progress in, in, in a lot of the, the, the closed source models. There was also this sort of proliferation early on of a lot of open source models that sort of, none were better than the other, and they just involved a lot of training kind of for companies, for lack of a better word, to just demonstrate that they could do it too. It's not clear that that's a valuable thing, right? I mean, why would you want to train your own language model from scratch if there are very good open source ones now? Will that continue? Maybe, maybe not. And I think there will be most likely consolidation, but I'm not quite sure how it will play out.

AI assessment note: “I think there will be most likely consolidation, but I'm not quite sure”

Q kind of, model ecosystems is just the commoditization of models. I remember, you know, a year ago, 18 months ago, it was so expensive, so hard, there were so few players. Now that there's so many, the commoditization, the reduction in cost, how do you expect this model landscape to play out, given the commoditization being really one of the fastest commoditizing technologies we seem to have seen in years?

A I think that it's been evolving so quickly between Uh, recent releases of open source models, uh, continued progress in, in, in a lot of the, the, the closed source models. There was also this sort of proliferation early on of a lot of open source models that sort of, none were better than the other, and they just involved a lot of training kind of for companies, for lack of a better word, to just demonstrate that they could do it too. It's not clear that that's a valuable thing, right? I mean, why would you want to train your own language model from scratch if there are very good open source ones now? Will that continue? Maybe, maybe not. And I think there will be most likely consolidation, but I'm not quite sure how it will play out.

AI assessment note: “I think there will be most likely consolidation, but I'm not quite sure”

Q When we think about optimizing the data that we have in terms of kind of value extraction, what could be done further to get further value from the data that we have?

A I don't really know, to be honest. I think this is a major open question right now in research. How do we extract the maximal information content? From these models, or sorry, from the data that we have. Um, but again, as I said, I, I don't think we're close to even extracting all the data that's available, so there's, when I look at this landscape, right, and we know that we aren't close to extracting the maximal information in sort of, you know, the closure set of all the data that we have available, and we aren't come, we have not come close to processing all the data that's available to us. The idea that somehow this is a, a recipe for models plateauing in performance Just doesn't jive to me with the reality of what we see.

AI assessment note: “I don't really know, to be honest. I think this is a major open question”

Q Alex Wang at scale said a brilliant line on the show. He said that, um, essentially we have a technology now that is more potentially dangerous and impactful than nuclear weapons. I mean, kind of talk about a sound bite. My question to you is if that is the case, or even partially the case, or even potentially the case, is there any world in which it should be open?

A Two issues there. One is AI as dangerous as nuclear weapons, and What does this imply about sort of the open release of certain models? Um, so I'll make two points on this. Um, I think the nuclear weapon analogy is actually not a great one, because nuclear weapons have one purpose, which is to, to destroy things. Maybe a better analogy is sort of nuclear technology, period, because it has the ability to create nuclear weapons, but it also has the ability to do things like provide power, uh, non, non, uh, CO₂ emitting power to Potentially a huge number of people, right? Um, and, and, you know, a lot of people are currently making a bet on, on nuclear as the, as the way we create carbon-free, carbon-free energy. But I think the analogy to nuclear weapons in particular is often overstated precisely because AI has many good uses. Nuclear weapons, arguably, they, they do one thing, and it's, it's not considered a good use, right? This is, this is, so this is a very different kind of technology there.

AI assessment note: “Two issues there. One is AI as dangerous as nuclear weapons, and What does”

Q Do you think we will see AI bring back a movement from large enterprises away from the cloud back to on-prem?

A I mean, I find this kind of, kind of interesting in a way because enterprises are all very happy to put their data in the cloud. You know, they, they, they all use cloud services to store their data, but then, oh, train on this there? No, no, can't do that. I think a lot of it comes honestly from kind of a misunderstanding about how this process works. Also, frankly speaking, I think it has to do with the fact that if you think about the model of just taking all your internal data and dumping it into a large language model, this is not tenable, right? You can't do this for a number of reasons. The most obvious one being the data has access rights, right? Not everyone gets access to all the data. And the default mode of language models is that if you, um, if you put train on some data, you can probably get it back out of the system, uh, if you want to enough. And so this doesn't work with, with the, the sort of the access controls people have in traditional, traditional data. I think these are the kind of the concerns. Now there are there, to be clear, there are very easy ways around this, right? So, so this is probably why RAG based systems are so Are so common here, and so, and probably will remain, even with the advent of fine-tuning availability, they're going to remain a useful paradigm. RAG is, for those that maybe haven't heard the term, it's retrieval augmented generatio…

AI assessment note: “enterprises are all very happy to put their data in the cloud”

Q When you think about AI safety, though, then, should the platforms themselves be the arbiters of justice of what's right and what's not right? You know, Twitter, Facebook, Reddit, and are they the ones that say, no, This is not allowed content.

A There are some things I believe that should not be shared on social media. And by the way, everyone else agrees with this too, right? There's obviously content that is outright considered illegal that you cannot post to social media. Everyone agrees on this. Um, everyone also agrees that, well, not everyone, but a lot of people also agree that in general, there should be a share, you know, there should not be a requirement to conform to certain ideologies and opinions if you want to express yourself on social media. And so, There's obviously a middle ground. You have to, and you have to toe the line here, and you have to adapt to the reality of the situation on the ground, and kind of go from there. I think, in many ways, and this is maybe what I was pointing out before, is that AI, when it comes to things like misinformation, it is not It did not invent misinformation, AI. It can argue there was misinformation and propaganda and this stuff long before there was AI. You can argue it's an accelerant to it, like it's an accelerant for everything, um, for a lot of things that we have, right? Um, but it did not invent these things, and my hope at least is that a lot of our existing social and economic and governmental structures Can continue to provide the same guidance they provided for a current kind of take on moderation and things like this, even in an AI world.

AI assessment note: “existing social and economic and governmental structures Can continue to provide the same guidance”

Q To what extent do you think the corporations that we mentioned are chasing AGI, super intelligence, versus making amazing products and leveraging AI to make them and make more money?

A Right. So, so this question is, is, I think, super interesting, and those are not actually mutually exclusive, to be, to be clear. One thing I'll also say is that the term AGI is thrown around a whole lot. I define AGI as a system that acts functionally equivalent to a close collaborator of yours over the course of about a year long project. So this is something that, you know, you would value as much as a close collaborator, you know, a student of mine or a colleague of mine over working on a project for a year. Um, it's fine if this is virtual, by the way, uh, I, you know, I think embodied AI is going to take a little bit longer to reach us, but Let's think about that definition of AI. Or maybe even more so, since we always think less of, I mean, I, I get this comment a lot. It's just like, well, AGI could automate this other people's job, but, but, but not mine. Let's think about me. AGI would be something that could automate everything that I do for the most part. Maybe, you know, besides the, the, the softer kind of emotional qualities that I, that I bring maybe. But AGI could automate everything that I functionally do in my job, uh, over a year. So, you know, everyone that comes to me, that comes to me right now to ask me things, uh, you on this podcast, you would just say, you know, I don't really care about talking with, with Zico. I'm just going to talk to the AGI inst…

AI assessment note: “those are not actually mutually exclusive, to be, to be clear.”

Q Do you think we will see AI bring back a movement from large enterprises away from the cloud back to on-prem?

A I mean, I find this kind of, kind of interesting in a way because enterprises are all very happy to put their data in the cloud. You know, they, they, they all use cloud services to store their data, but then, oh, train on this there? No, no, can't do that. I think a lot of it comes honestly from kind of a misunderstanding about how this process works. Also, frankly speaking, I think it has to do with the fact that if you think about the model of just taking all your internal data and dumping it into a large language model, this is not tenable, right? You can't do this for a number of reasons. The most obvious one being the data has access rights, right? Not everyone gets access to all the data. And the default mode of language models is that if you, um, if you put train on some data, you can probably get it back out of the system, uh, if you want to enough. And so this doesn't work with, with the, the sort of the access controls people have in traditional, traditional data. I think these are the kind of the concerns. Now there are there, to be clear, there are very easy ways around this, right? So, so this is probably why RAG based systems are so Are so common here, and so, and probably will remain, even with the advent of fine-tuning availability, they're going to remain a useful paradigm. RAG is, for those that maybe haven't heard the term, it's retrieval augmented generatio…

AI assessment note: “enterprises are all very happy to put their data in the cloud. You know,”