The Exchanges

Q It's a, it's a Friday afternoon. We're gonna have a fun chat. Fuck the normal stuff. Okay, so you said there about not being ready for, like, traditional adoption. I think a lot of the general public would say, absolutely, it's, it's, it's cool. What makes it not ready for general adoption, do you think?

A Yeah. So there are a couple of just really big issues. Um, hallucination is very big one. These models make things up with very high confidence. Um, attribution. We don't know why they're saying what they're saying. We can't really trace it back to anything. Um, there's compliance issues, so we can't really remove information from them, uh, which is kind of tricky from a GDPR perspective. For example, uh, we can't revise information. We can't keep it up to date. Uh, there's massive data privacy issues where you have to send your very valuable company data. If you're an enterprise, you have to send that to somebody else's servers. These models are also quite inefficient still, so you can make them much faster. And so what we are building at contextual is a different kind of language model. Um, and, uh, so we're really thinking about this as the next generation of language models, where we think about it from first principles for enterprise use cases. And what that means is that we want to solve all of these problems by being a bit smarter about the architecture and the architecture. We're specifically basing it on this retrieval augmented generation, which is something that, uh, me and my colleagues at fair came up with in. And what you do there is you decouple the memory from the generative capacity of the large language model, and this allows you to ground the generations from…

AI assessment note: “So there are a couple of just really big issues. Um, hallucination is very big one.”

Q In terms of, like, growing with the capabilities of the models, uh, we've seen size of model matter less and less, it would seem. How do you think about the importance of size of model today, and does it matter as much as it used to, and will it matter even less with every year, month, day?

A Yeah, great question. I think Sam Altman had this interesting quote where he was saying that he thought models would stop growing in size. Um, and, uh, so that, that GPT-IV kind of hit this ceiling. And I think, I think that's probably right, but not really because size doesn't matter. It's just that data size matters even more than model size. And I think the Lama paper, uh, out of, out of Meta really brilliantly showed this. Um, where if you train a smaller model on more data for longer, then you get a better model. So you get more bang for your buck if you train it on more data rather than having more parameters. But in an ideal world, if you had infinite compute budget and infinite data, then you would train the biggest possible model because that's the most likely to give you sort of emergent capabilities as we call them, uh, in the field.

AI assessment note: “data size matters even more than model size”

Q What can you do? Can you just talk us through that? What does that separation mean?

A Yeah. So, so traditionally in, in cloud deployments, you talk about the data plane and the control plane. Um, and so control really is okay. You're, you're a startup or a company, and you want to be able to deploy models, uh, and things like that to your customer's VPC, virtual private cloud, but you don't want any of their data leaving their VPC, um, because it's their data. So that's the separation between the data plane and the control plane. So in this new setting where we have language models, there there's this model plane, and it's kind of unclear where to put it. So you could put the model inside. The customer's VPC, um, and, and then you get full data privacy basically, but you have no control over what that model is doing at all, and you get no feedback, you get no learning. So you want to find interesting hybrids where you can respect data privacy, keep the data plane inside the customer's VPC, but put the model somewhere else. So you can do that if you have a decoupling between the retrieval part and the generative part, which is what we are building.

AI assessment note: “So you want to find interesting hybrids where you can respect data privacy”

Q In terms of, like, growing with the capabilities of the models, uh, we've seen size of model matter less and less, it would seem. How do you think about the importance of size of model today, and does it matter as much as it used to, and will it matter even less with every year, month, day?

A Yeah, great question. I think Sam Altman had this interesting quote where he was saying that he thought models would stop growing in size. Um, and, uh, so that, that GPT-IV kind of hit this ceiling. And I think, I think that's probably right, but not really because size doesn't matter. It's just that data size matters even more than model size. And I think the Lama paper, uh, out of, out of Meta really brilliantly showed this. Um, where if you train a smaller model on more data for longer, then you get a better model. So you get more bang for your buck if you train it on more data rather than having more parameters. But in an ideal world, if you had infinite compute budget and infinite data, then you would train the biggest possible model because that's the most likely to give you sort of emergent capabilities as we call them, uh, in the field.

AI assessment note: “It's just that data size matters even more than model size.”

Q Okay, sorry, I, I think this is, like, fascinating when you think about, kind of, the Moody's and the S&P opportunity here. The question is, though, that I, I still don't quite understand. How do we know a yardstick for progress or measurement? What is the right way to approach AI measurement and effectiveness?

A Yeah, so, so the standard way to do it and, and there are, are great benchmarks for this, right? So there's a Stanford helm project, the holistic evaluation of language models. Um, and there's a long tradition in NLP around like glue and super glue and, and these benchmarks where you look on static test sets, how good language models or any other kind of NLP model, uh, how good they are, uh, on this static test set that's held out. But I've been arguing for a long time that that's just completely wrong anyway, and we need to do something that's much more dynamic. Uh, so I've been involved in this DynaBench project DynaBench.org. Um, and the idea there is that you cannot really have a static test set because people can just overfit on that test set. It doesn't really give you the full signal you want. So ideally what you want is to see how easy is it for an adversarial person to mess with your model. And the harder that is, the better your model is. Um, and, and so it used to be very easy to, to come up with adversarial attacks where these models would just completely mess up. Uh, and it's getting harder and harder over time. So, so that sort of success rate of an adversarial attacker, that's something that, that you can keep evaluating over time. So we need humans to evaluate these models by trying to break them.

AI assessment note: “we need to do something that's much more dynamic. Uh, so I've been involved”

Q up, maybe a Balenciaga hoodie, probably not given the price, but put a hugging face. And so it makes it much more Personalized, real, cool. I thought it was really interesting actually, just kind of, I've never heard that before. Okay, so we have Hugging Face and Experience. Where does contextual come from? What was that? Aha, I've got to do this. This is now the idea and the time.

A Yeah, so this really started at the beginning of the year. Uh, so me and my co-founder, Amanpreet Singh, um, who I worked with first at Facebook and then at Hugging Face. Um, and he really is one of the smartest people I know. He's really an incredible guy. And we were talking about Uh, what to do next? And, uh, we, we just saw this great need after ChatGPT had gone viral. We saw this, this kind of great excitement in the world, but at the same time, a lot of disappointment, uh, about it not being quite ready yet for real world adaption, uh, in, in enterprises, uh, where you actually want to use this technology. So we decided that now really is the right time to, to build a company to try to tackle that. Um, and we think it's still very early innings in the game, so I, I think a lot of people sometimes think that it's, uh, you know, the game has been played, but it's just getting started.

AI assessment note: “so this really started at the beginning of the year”

Q Okay, sorry, I, I think this is, like, fascinating when you think about, kind of, the Moody's and the S&P opportunity here. The question is, though, that I, I still don't quite understand. How do we know a yardstick for progress or measurement? What is the right way to approach AI measurement and effectiveness?

A Yeah, so, so the standard way to do it and, and there are, are great benchmarks for this, right? So there's a Stanford helm project, the holistic evaluation of language models. Um, and there's a long tradition in NLP around like glue and super glue and, and these benchmarks where you look on static test sets, how good language models or any other kind of NLP model, uh, how good they are, uh, on this static test set that's held out. But I've been arguing for a long time that that's just completely wrong anyway, and we need to do something that's much more dynamic. Uh, so I've been involved in this DynaBench project DynaBench.org. Um, and the idea there is that you cannot really have a static test set because people can just overfit on that test set. It doesn't really give you the full signal you want. So ideally what you want is to see how easy is it for an adversarial person to mess with your model. And the harder that is, the better your model is. Um, and, and so it used to be very easy to, to come up with adversarial attacks where these models would just completely mess up. Uh, and it's getting harder and harder over time. So, so that sort of success rate of an adversarial attacker, that's something that, that you can keep evaluating over time. So we need humans to evaluate these models by trying to break them.

AI assessment note: “we need humans to evaluate these models by trying to break them.”

Q What's the strongest belief that you had, which turned out to be wrong?

A The strongest belief I had that turned out to be wrong, uh, is that I really underestimated how important scale is in artificial intelligence. Um, so I, and I think this is really one of the things that open AI has excelled at, um, is that if you throw an order of magnitude more compute and data at AI, uh, systems, then they just become much, much better. And if you keep that scaling up, you have these scaling laws that we know about now. I, I really underestimated this. And, and for a long time, I was just saying like, oh, yeah, look at these, uh, silly, uh, open AI researchers. They're, they're just scaling things. They're not inventing new algorithms. That's not cool. Uh, and I was very, very wrong.

AI assessment note: “I really underestimated how important scale is in artificial intelligence.”

Q up, maybe a Balenciaga hoodie, probably not given the price, but put a hugging face. And so it makes it much more Personalized, real, cool. I thought it was really interesting actually, just kind of, I've never heard that before. Okay, so we have Hugging Face and Experience. Where does contextual come from? What was that? Aha, I've got to do this. This is now the idea and the time.

A Yeah, so this really started at the beginning of the year. Uh, so me and my co-founder, Amanpreet Singh, um, who I worked with first at Facebook and then at Hugging Face. Um, and he really is one of the smartest people I know. He's really an incredible guy. And we were talking about Uh, what to do next? And, uh, we, we just saw this great need after ChatGPT had gone viral. We saw this, this kind of great excitement in the world, but at the same time, a lot of disappointment, uh, about it not being quite ready yet for real world adaption, uh, in, in enterprises, uh, where you actually want to use this technology. So we decided that now really is the right time to, to build a company to try to tackle that. Um, and we think it's still very early innings in the game, so I, I think a lot of people sometimes think that it's, uh, you know, the game has been played, but it's just getting started.

AI assessment note: “we decided that now really is the right time to, to build a company”

Q It's a, it's a Friday afternoon. We're gonna have a fun chat. Fuck the normal stuff. Okay, so you said there about not being ready for, like, traditional adoption. I think a lot of the general public would say, absolutely, it's, it's, it's cool. What makes it not ready for general adoption, do you think?

A Yeah. So there are a couple of just really big issues. Um, hallucination is very big one. These models make things up with very high confidence. Um, attribution. We don't know why they're saying what they're saying. We can't really trace it back to anything. Um, there's compliance issues, so we can't really remove information from them, uh, which is kind of tricky from a GDPR perspective. For example, uh, we can't revise information. We can't keep it up to date. Uh, there's massive data privacy issues where you have to send your very valuable company data. If you're an enterprise, you have to send that to somebody else's servers. These models are also quite inefficient still, so you can make them much faster. And so what we are building at contextual is a different kind of language model. Um, and, uh, so we're really thinking about this as the next generation of language models, where we think about it from first principles for enterprise use cases. And what that means is that we want to solve all of these problems by being a bit smarter about the architecture and the architecture. We're specifically basing it on this retrieval augmented generation, which is something that, uh, me and my colleagues at fair came up with in. And what you do there is you decouple the memory from the generative capacity of the large language model, and this allows you to ground the generations from…

AI assessment note: “hallucination is very big one. These models make things up with very high confidence.”

Q there will be many more startup income, startup language model companies? Like when we look at what we have today, there are kind of the core incumbents, and I'd probably include even your, well, I would definitely include your open AIs, but even your kind of anthropics of the world in there, but like startup startup, do you think we already have our incumbent model companies set and fully baked?

A Well, I don't think so. Otherwise I wouldn't have started this new company. Um, I, I think. This technology is really going to change the world and every aspect of it, uh, which means that it's a giant market and there's a lot of room for lots of companies to do innovative things. Um, so there are definitely a, a couple of incumbents, um, but they are also focused on very specific parts of the market. So if, if you look at anthropic and open AI, I think they're really, uh, chasing for this idea of AGI and they're, they're relatively consumer facing. Um, and, and if you. Uh, focus less on this idea of artificial general intelligence, and you want to have something a bit more like artificial specialized intelligence where you just need the model to do what it needs to do. You don't need it to know about Shakespeare or quantum mechanics or things like that. You just need it to solve your business's problem. In that case, I, I think there's still a lot of room, uh, for innovation, and that's where we are trying to innovate.

AI assessment note: “Well, I don't think so. Otherwise I wouldn't have started this new company.”

Q But, I, I don't understand. Who's more advantaged in that case? Is it incumbents, because they already have existing massive data modes?

A It depends on where the data comes from. I, I think income is definitely have an advantage there. Uh, but only some of them, a lot of the data is just freely available on the Internet, right? So the Lama model was not trained on any proprietary data. It was just trained on open data on the web, and there's a lot more data to be had there. And as, as a society, we're generating a ton of data every day to add to that big pile of data. Uh, so you can really train very high quality language models just on public. Uh, data on the, on the web. But I think if you look at the secret sauce to a lot of these other models, like why is GPT-IV so awesome? A part of that, uh, is that they, they went through, uh, enormous, uh, lengths to, to get like special data that nobody else has. So allegedly they did this whisper project where they're very good at transcribing audio because that would allow them to transcribe like all of the podcasts in the world, which gives you very high quality, uh, Language. So if you can train on that language, but nobody else has it, that puts you in a position of advantage.

AI assessment note: “I think income is definitely have an advantage there. Uh, but only some of them”

Q What can you do? Can you just talk us through that? What does that separation mean?

A Yeah. So, so traditionally in, in cloud deployments, you talk about the data plane and the control plane. Um, and so control really is okay. You're, you're a startup or a company, and you want to be able to deploy models, uh, and things like that to your customer's VPC, virtual private cloud, but you don't want any of their data leaving their VPC, um, because it's their data. So that's the separation between the data plane and the control plane. So in this new setting where we have language models, there there's this model plane, and it's kind of unclear where to put it. So you could put the model inside. The customer's VPC, um, and, and then you get full data privacy basically, but you have no control over what that model is doing at all, and you get no feedback, you get no learning. So you want to find interesting hybrids where you can respect data privacy, keep the data plane inside the customer's VPC, but put the model somewhere else. So you can do that if you have a decoupling between the retrieval part and the generative part, which is what we are building.

AI assessment note: “So that's the separation between the data plane and the control plane.”

Q How important is proprietary data? The main reason I would say why VCs are turning down startup AI companies is because they Do not have a proprietary data set to operate against, and they are defined as like a thin layer of generative AI on top of a foundational model. How important is proprietary data do you think for startups innovating in the space?

A If you want to build a deep tech AI startup, then you really want to get a big data flywheel going. You want to start with a lot of data and then have a way to generate lots more data and that data is going to be your moat. But I think one of the interesting things about these large language models is that they're incredibly sample efficient or data efficient. So you can do cool things with them with relatively little data that just previously just wasn't possible. That unlocks all kinds of possibilities that just didn't exist even a couple of years ago. So on the one hand, yes, you need lots of data if you want to build like big AI first things, but at the same time, if you want to do a startup that builds on top of this technology, you need very little data. To get started. Bit of a tangent. But one of the use cases I've been seeing now for GPT-IV is actually that people are using it to generate data and then they're training on that data with cheaper models. So GPT-IV might end up disrupting, not knowledge workers necessarily, but it might just disrupt like Mechanical Turk and is just an annotator on steroids. And you can use all of that data to get much more custom models that you can then deploy very cheaply on specialized use cases. That's a quite interesting development.

AI assessment note: “if you want to do a startup that builds on top of this technology, you need very little data.”

Q Oh, it was very, very kind of you. I literally paid you 25,000 dollars to say that, but, um, um, but my question to you is, it's such a hot space and there's very few people who've actually been in it for a while. You are one of them. How did you first make your way into the world of ML and NLP first?

A Yeah, my, my journey has been a little bit unusual actually. So, um, uh, when I was in high school in the Netherlands, Um, I wanted to be a cool kid during the day, but at night I was secretly fascinated by computers. Um, so I started off as a script kitty wanting to hack other people's computers and figured out that if I really wanted to do that, I had to learn to code. So I taught myself to code, uh, figured I needed to understand operating systems. So I made my own operating system, uh, with bootloader and everything, uh, when I was 16. Uh, so then by the time, uh, I had to go to College and, and go study something. I, I thought I already knew everything about computer science. Uh, so I decided to study philosophy instead. Um, and, and so that was really a very, uh, radical departure from what I had been interested in at the time. Um, but it was fascinating, uh, learning a lot about the mind and language and things like that. It's, I, I use it still every day, I think. Um, but then, uh, at some point in my career, it became Clear that I had to start making money. So I needed a real job and philosophy is not really a real job. And I did some logic in between foundations of math, which is also not really a real job. So I decided to study computer science after all. Uh, so I went to Cambridge, uh, in the UK, uh, had a fantastic time there. Um, and, and so that's really where I …

AI assessment note: “so that's really where I started doing NLP, natural language processing.”

Q Can I ask you, which company do you think has the best data acquisition flywheel? When you look at them today, who do you admire and respect most?

A Open AI. Um, so maybe that's not the answer you expected, but so it's just an incredible company and they've really shown the world what's possible. Um, and they haven't even really trained as far as I know on the data that comes out of ChatGPT going viral, right? So they had ChatGPT, it went viral. This led to this giant, giant data mode that they haven't even really used, used yet. Um, so I, I think in terms of data modes and, and maybe you, you've seen this come by actually, there was this Google memo, uh, from an internal Google employee who, who had written that open AI and Google have no mode. I, I think for me, for me as a AI researcher, when I read that memo, I was like, this person has no idea what they're talking about.

AI assessment note: “Open AI. Um, so maybe that's not the answer you expected”

Q to ask. You mentioned the word hallucinations there. I had Imad at Stability on the show, and he said, Hallucinations are a feature, not a bug. Which I thought was a very tweetable statement. Yeah, that's a great quote. But I didn't quite understand it. Do you agree hallucinations are a feature, not a bug? He is the only person to have said this on the show, to be clear.

A So, um, I, I think it's a, it's a great quote, uh, but as always, it's a bit more nuanced than that, right? So I think in some cases it is a feature. If you want to use a language model for creative writing, uh, and if you want it to be really, really creative, then you probably want it to hallucinate. So in a way it's a spectrum of groundedness and hallucination where, uh, if you really care about the language model doing the right thing and you want to deploy it in an enterprise critical situation, Then you really don't want it to be creative. You don't want it to hallucinate. You just want it to do what it has to do. But if you want to use it for a creative writing exercise, then sure you can have it hallucinate because you're going to revise whatever it gives you anyway.

AI assessment note: “in some cases it is a feature. If you want to use a language model”

Q But, I, I don't understand. Who's more advantaged in that case? Is it incumbents, because they already have existing massive data modes?

A It depends on where the data comes from. I, I think income is definitely have an advantage there. Uh, but only some of them, a lot of the data is just freely available on the Internet, right? So the Lama model was not trained on any proprietary data. It was just trained on open data on the web, and there's a lot more data to be had there. And as, as a society, we're generating a ton of data every day to add to that big pile of data. Uh, so you can really train very high quality language models just on public. Uh, data on the, on the web. But I think if you look at the secret sauce to a lot of these other models, like why is GPT-IV so awesome? A part of that, uh, is that they, they went through, uh, enormous, uh, lengths to, to get like special data that nobody else has. So allegedly they did this whisper project where they're very good at transcribing audio because that would allow them to transcribe like all of the podcasts in the world, which gives you very high quality, uh, Language. So if you can train on that language, but nobody else has it, that puts you in a position of advantage.

AI assessment note: “It depends on where the data comes from. I, I think income is definitely”

Q How important is proprietary data? I'm a VC for my sins. The main reason I would say why VCs are turning down startup AI companies is because they do not have a proprietary data set to operate against, and they are defined as like a thin layer of generative AI on top of a Foundational model. How important is proprietary data, do you think, for startups innovating in the space?

A It really depends on, on the specific startup. Uh, so if, if you want to build a deep tech AI startup, then you really want to get a big data flywheel going. So, uh, you want to start with a lot of data and then have a way to generate lots more data and that data is going to be your mode. But I think one of the interesting things about these large language models is that they're incredibly sample efficient or data efficient. So you can do cool things with them with relatively little data. Uh, that just previously just wasn't possible. Uh, so, uh, that unlocks all kinds of possibilities that just didn't exist even a couple of years ago. So on the one hand, yes, you need lots of data if you want to build like big AI first things, but at the same time, if you want to do a startup that builds on top of this technology, you need very little data to get started. Um, and actually one, one interesting, maybe a little bit of a tangent. But one of the use cases I've been seeing now for GPT-IV is actually that people are using it to generate data and then they're training on that data with cheaper models. Um, uh, and so, so GPT-IV might end up disrupting, not, um, like knowledge workers necessarily, but it might just disrupt like mechanical Turk and is just a, an annotator on steroids. And you can use all of that data to get much more custom models that you can then deploy very cheaply on…

AI assessment note: “It really depends on, on the specific startup.”

Q Can I be a bit savage? When you look at all the VC fundraisers, do you go, oh my god, these guys have no idea what they're doing when they're investing in X startup or Y company? Do you look at it and go, this is getting crazy?

A Um, not so much. I think some of the rounds were pretty big, but I think it's also justified just because this stuff is really going to change the world. And so one, one bet, if it's right, has massive payoff. Um, so there, there's this narrative, I think in the VC community that Um, that there are these crazy rounds happening, but I think they're happening for good reason. Uh, so I haven't really seen any, any companies come by where I was like, wow, like, why are they getting this much money? There, there's a few of them where I thought, okay, they, they really have to live up to massive expectations now, and they need to actually start making real revenue now. Um, Because, uh, at, at some point there's going to be a disillusionment with the technology and then funding might dry up and then these places are really in trouble.

AI assessment note: “Um, not so much. I think some of the rounds were pretty big”

Q How important is proprietary data? I'm a VC for my sins. The main reason I would say why VCs are turning down startup AI companies is because they do not have a proprietary data set to operate against, and they are defined as like a thin layer of generative AI on top of a Foundational model. How important is proprietary data, do you think, for startups innovating in the space?

A It really depends on, on the specific startup. Uh, so if, if you want to build a deep tech AI startup, then you really want to get a big data flywheel going. So, uh, you want to start with a lot of data and then have a way to generate lots more data and that data is going to be your mode. But I think one of the interesting things about these large language models is that they're incredibly sample efficient or data efficient. So you can do cool things with them with relatively little data. Uh, that just previously just wasn't possible. Uh, so, uh, that unlocks all kinds of possibilities that just didn't exist even a couple of years ago. So on the one hand, yes, you need lots of data if you want to build like big AI first things, but at the same time, if you want to do a startup that builds on top of this technology, you need very little data to get started. Um, and actually one, one interesting, maybe a little bit of a tangent. But one of the use cases I've been seeing now for GPT-IV is actually that people are using it to generate data and then they're training on that data with cheaper models. Um, uh, and so, so GPT-IV might end up disrupting, not, um, like knowledge workers necessarily, but it might just disrupt like mechanical Turk and is just a, an annotator on steroids. And you can use all of that data to get much more custom models that you can then deploy very cheaply on…

AI assessment note: “It really depends on, on the specific startup. Uh, so if, if you want”

Q of a black box. I had someone on the show the other day, and they said, open or closed, you still don't really know what's going on in the core foundational model layer. The open or closed is not really the point. You still don't know. Is that true? And how do we think about actual true transparency of knowing what's going on and why it's producing what it is?

A Yes. So we're not going to be able to, um, really know why a neural net, what does what it does at the scale that neural networks operate, uh, at. So this is kind of like your own brain, right? Like, so I, I think your behavior is, is relatively predictable. Um, uh, so that goes for every human, right? We all like can predict each other's actions, but I have no idea what's going on in your brain and I will never know. There's no way I can know. The only, the only way I can kind of find out is by asking you. Um, but if you train the architecture the way we are training it right now, then at least you make sure that the model has learned to rely on the information that it finds. Um, and that gives you much stronger attribution than if it's just predicting the next word based on what it has seen before, uh, because it doesn't have this ability from birth basically to find relevant information and ground its generation, uh, on that, uh, uh, For the thing it found.

AI assessment note: “Yes. So we're not going to be able to, um, really know”

Q to ask. You mentioned the word hallucinations there. I had Imad at Stability on the show, and he said, Hallucinations are a feature, not a bug. Which I thought was a very tweetable statement. Yeah, that's a great quote. But I didn't quite understand it. Do you agree hallucinations are a feature, not a bug? He is the only person to have said this on the show, to be clear.

A So, um, I, I think it's a, it's a great quote, uh, but as always, it's a bit more nuanced than that, right? So I think in some cases it is a feature. If you want to use a language model for creative writing, uh, and if you want it to be really, really creative, then you probably want it to hallucinate. So in a way it's a spectrum of groundedness and hallucination where, uh, if you really care about the language model doing the right thing and you want to deploy it in an enterprise critical situation, Then you really don't want it to be creative. You don't want it to hallucinate. You just want it to do what it has to do. But if you want to use it for a creative writing exercise, then sure you can have it hallucinate because you're going to revise whatever it gives you anyway.

AI assessment note: “I think in some cases it is a feature. If you want to use”

Q Can I be a bit savage? When you look at all the VC fundraisers, do you go, oh my god, these guys have no idea what they're doing when they're investing in X startup or Y company? Do you look at it and go, this is getting crazy?

A Um, not so much. I think some of the rounds were pretty big, but I think it's also justified just because this stuff is really going to change the world. And so one, one bet, if it's right, has massive payoff. Um, so there, there's this narrative, I think in the VC community that Um, that there are these crazy rounds happening, but I think they're happening for good reason. Uh, so I haven't really seen any, any companies come by where I was like, wow, like, why are they getting this much money? There, there's a few of them where I thought, okay, they, they really have to live up to massive expectations now, and they need to actually start making real revenue now. Um, Because, uh, at, at some point there's going to be a disillusionment with the technology and then funding might dry up and then these places are really in trouble.

AI assessment note: “Um, not so much. I think some of the rounds were pretty big, but I think it's also justified”

Q Can I ask in terms of like the proprietary data element, pre-trained data changes a lot. Can you just help anyone who doesn't know understand what is pre-trained data and how does it change the game for a lot of companies that don't have existing data modes?

A Yeah, so I, I think maybe it's useful to kind of go through the, the steps if you want to build your own chat GPT, like what do you need? And so the first thing you need is a core pre-trained model, and this tends to be just trained on the web. The, the task you're training it on is just next word prediction. Then once you have that core model, then you want to do supervised fine tuning. So essentially you want to fix the user interface to that model because the model doesn't really. Uh, know how to follow instructions, for example. So you want the model to listen to you, but it has only been trained on predicting the next word, so it doesn't really know how to do that. So that supervised fine tuning, that's also proprietary data, uh, if you want, um, you can, you can get a much better model out of that. And then the final step is RLHF, reinforcement learning from human feedback, where you get this feedback loop to make the model even better for your specific use case, even if you don't have signal At the word level, you just have signal at the sequence level. So you can tell it like, okay, that was a good response, or that wasn't a good response, but you can tell it like, what did you do wrong necessarily? So if you, if you go through those three steps, then you get a ChatGPT. It's as easy as that. Um, but obviously there's a, the devil's in the details.

AI assessment note: “the first thing you need is a core pre-trained model, and this tends to be just trained on the web”

Q How important is proprietary data? The main reason I would say why VCs are turning down startup AI companies is because they Do not have a proprietary data set to operate against, and they are defined as like a thin layer of generative AI on top of a foundational model. How important is proprietary data do you think for startups innovating in the space?

A If you want to build a deep tech AI startup, then you really want to get a big data flywheel going. You want to start with a lot of data and then have a way to generate lots more data and that data is going to be your moat. But I think one of the interesting things about these large language models is that they're incredibly sample efficient or data efficient. So you can do cool things with them with relatively little data that just previously just wasn't possible. That unlocks all kinds of possibilities that just didn't exist even a couple of years ago. So on the one hand, yes, you need lots of data if you want to build like big AI first things, but at the same time, if you want to do a startup that builds on top of this technology, you need very little data. To get started. Bit of a tangent. But one of the use cases I've been seeing now for GPT-IV is actually that people are using it to generate data and then they're training on that data with cheaper models. So GPT-IV might end up disrupting, not knowledge workers necessarily, but it might just disrupt like Mechanical Turk and is just an annotator on steroids. And you can use all of that data to get much more custom models that you can then deploy very cheaply on specialized use cases. That's a quite interesting development.

AI assessment note: “if you want to do a startup that builds on top of this technology, you need very little data”

Q love that. Um, no, I totally get you. You mentioned regulation there. I think a big question for me is, like, I don't think the chasm has ever been greater between private, like, company knowledge, especially around AI, specifically around AI, and then also the regulator's knowledge, which is significantly, ah, behind. How can effective regulation be set with such a large chasm between private sector knowledge and regulator knowledge?

A Yeah, we have to, uh, invest a lot in educating regulators. Uh, and I, I think the AI community has been terrible at this. And, um, I think the broader populace just needs to understand much better what AI is and what it can do and what it can't do. Um, and, and I think Uh, it, it's been slightly self-interest driven, I think, uh, in that a lot of folks in, in AI have just wanted to keep the technology for themselves. And that's why they haven't really invested in, in educating, uh, the rest of society. Uh, so I, I think that that's really a, a huge issue. And so there's a bit of a side point there, but I think the people who tend to write the regulation, they generally don't really understand technology all that way, all that much anyway. Um, so if you look at like the Senate hearings with Zuck, where the editors were asking him questions about the social network algorithm or whatever, and they didn't know what an algorithm was. Right. So the, that, that didn't you love it?

AI assessment note: “we have to invest a lot in educating regulators”

Q I actually spoke to Leon as part of the prep for my interview with Jan. Amazing, amazing person. But I do want to talk about that five years that you spent at, kind of, Facebook's AI research team. It's such a transformational team, as you said, with incredible individuals like Jan and Leon. What are your biggest takeaways from that experience, and how did it impact how you think today?

A It's really a one of a kind place. Uh, especially when I, when I just joined, uh, they had these amazing people who I really respected, uh, a ton like Jason Weston and Thomas Mikulov really kind of the godfathers of modern deep learning for NLP. I, I learned so much there mostly around how to focus your research direction. So I think initially I was doing all kinds of weird stuff. Um, and it took me a while to figure out that having a very clear Real world application for the research that you're doing makes it much more valuable than, than going off on a tangent and maybe being a bit too far ahead of the rest of the field. Um, so, um, yeah, it's really a special place. And I think actually they don't get enough credit for, uh, the impact that they've had on the world. So, and I, I mean like Facebook or meta in general, actually. So, um, almost every web app in the world runs on react, which is, uh, an open source, uh, Uh, project coming out of meta.

AI assessment note: “I learned so much there mostly around how to focus your research direction.”

Q on the show who we discussed earlier, obviously a very big proponent of kind of open models. Um, it's kind of a very big debate, obviously between open versus closed. Where do you sit in terms of the model that rules for the next five to 10 years? And is it different for the model that rules for the next five years versus that that rules for the next 10?

A So the way I think about the language model space is kind of as a pyramid. So at the top of the pyramid, we have these Frontier models. Uh, so these are GPT four and, and tropic models and things like that, that are just much better than everything else, but also much more expensive and much bigger than everything else. Um, and then at the bottom of the pyramid, you have open source models. Anybody can train on them. Anybody can fine tune them on their data. Uh, there's, you can run them on your phone now and on your laptop and things like that. So that's a, that's a very, uh, fruitful area for research. But I think the most interesting part is kind of the middle piece of that pyramid where you have the, the most bang for your buck. Uh, so that's from a business perspective, the most interesting part where you have Mid-sized models that have capabilities that you don't really see at this bottom of the pyramid, um, that you can monetize it in various ways. So I, I think it's not gonna be the case that there's just one model that wins everything. It's going to be lots of models at different parts, uh, uh, different layers of, of this pyramid being used for different kinds of applications. So if you have very strong AGI requirements, you probably want to have a frontier model. If you, Care about it a bit less. Maybe you want to have artificial specialized intelligence. If you care…

AI assessment note: “it's not gonna be the case that there's just one model that wins everything”

Q I actually spoke to Leon as part of the prep for my interview with Jan. Amazing, amazing person. But I do want to talk about that five years that you spent at, kind of, Facebook's AI research team. It's such a transformational team, as you said, with incredible individuals like Jan and Leon. What are your biggest takeaways from that experience, and how did it impact how you think today?

A It's really a one of a kind place. Uh, especially when I, when I just joined, uh, they had these amazing people who I really respected, uh, a ton like Jason Weston and Thomas Mikulov really kind of the godfathers of modern deep learning for NLP. I, I learned so much there mostly around how to focus your research direction. So I think initially I was doing all kinds of weird stuff. Um, and it took me a while to figure out that having a very clear Real world application for the research that you're doing makes it much more valuable than, than going off on a tangent and maybe being a bit too far ahead of the rest of the field. Um, so, um, yeah, it's really a special place. And I think actually they don't get enough credit for, uh, the impact that they've had on the world. So, and I, I mean like Facebook or meta in general, actually. So, um, almost every web app in the world runs on react, which is, uh, an open source, uh, Uh, project coming out of meta.

AI assessment note: “I learned so much there mostly around how to focus your research direction.”