The Exchanges

Q I'm not being rude, but could you not just buy, I know you haven't, but I could, could I not just do it, do a book and spend 75,000 dollars and be a New York Times bestseller then?

A So people do that all the time. Um, and, um, the way the New York Times has a small cabal of people who refuse to talk about how they do this. So they use the number ranking, but then they also try and exclude bulk buys. So they actually try and cut that out. So those you'll notice there's a little dagger next to the name of, uh, of, uh, companies, the best seller list that they think that they're including, but they still had potential bulk buys. I actually got the little dagger on mine because a company Um, bought 500 copies, which wasn't the main reason for the list, but they would have found that suspicious. Um, so they're trying to filter that out by hand. Um, so, but yes, you can often buy your way into the list, and people do, do that all the time.

AI assessment note: “yes, you can often buy your way into the list”

Q You said where the money is. I loved an analogy that you've said before. And it's, you said a lot of people use the analogy of picks and shovels and the gold rush. You said that's maybe not such a good analogy, and that the steam train was more apt. Why do you believe that it's not a good analogy, and why is the steam train more apt, Ethan?

A Analogies are really powerful, and we have very bad ones in AI. VC people get taken in by this, right? So it's like, I hear you want to sell picks and shovels, and first of all, I don't know, a hundred percent, like, everyone defines that slightly differently. They're like, oh, no, no, you want to sell compute, you want to sell, you know, you want to sell the tools that help people scale up and pick their, you know. That's actually not, you know, first of all, it's unclear what the analogy is, but the second deeper problem of this is that that isn't actually how a new technology spreads across an organization. You don't want to sell picks and shovels of the people trying to mine gold. What you want to do is figure out how to get them to use this new technology, which doesn't have a gold rush analogy at all, right? Instead the steam power and the steam power, the secret was not James Watts's steam engine, which was important, right? Huge breakthrough. Um, two interesting things, by the way, things didn't really take off until Watts patents expired, uh, and it can be openly adapted, but the real value of the steam engine came from having skilled artisans in your factory who said, I've got this thing that can make power go back and forth. How do I create the gearing to connect that to my, you know, my spinning Jenny, my ammunition manufacturing machine, my bottle shaping tool, And…

AI assessment note: “the real value of the steam engine came from having skilled artisans in your factory”

Q I have to, you said so many great things there. Would you tell your students today that they have to move to the Valley if they want to increase their chances of winning?

A That's an empirical result from a bunch of studies. Companies that, you know, there's been a study of Israeli companies in the Valley, New York companies. Like, it's just, the issue is, is that that's where the connections are, and it turns out Zoom only gets you so far. I mean, the average distance, at least pre-pandemic, I would, I'd be surprised if it actually changed. Um, the average distance between a VC and a company to invest in is about 40 miles. Like, that's because when you look at what, where VCs spend their time, it's networking and it's monitoring. It's networking with other, with, you know, and learning about companies, and then it's monitoring the portfolio companies. And that's much easier when you're local. Zoom doesn't let you do monitoring the same way. In fact, when a direct flight is added between SFO and another city, the VC investment in that city goes up. Because it's just getting, it's easier to fly there and help do, ah, and do monitoring there. It's a local business, right? Everyone's like, oh, it's global. It's connected. It's a local business.

AI assessment note: “That's an empirical result from a bunch of studies.”

Q Now for anyone that doesn't know your work, can you just give a 62nd intro on your work and how you've become much more well-known in the last few years?

A I'm a former entrepreneur myself. So the startup company I, I helped co-found, um, invented the paywall. So I still feel like I'm, I'm trying to make up for that in the late nineties. Um, so just trying to pay back after, after, after that. Um, but then I, I've been a professor of entrepreneurship. I got trained in MIT and then I've been at Wharton ever since, you know, I do a lot of work on teaching and thinking about, you know, research on how entrepreneurs become successful, but it also had this side gig of thinking about AI and teaching for a long time. So I worked at the media lab, uh, with a guy named Marvin Minsky, who's one of the founders of AI. And I was like the non-technical person there who was like trying to translate what the lab was doing for the world. And then I've been building tools for how do we teach entrepreneurship at scale? Because it turns out it really matters. Little bits of entrepreneurship training make a huge difference in people's lives. And we've been playing with AI and other tools. So when AI sort of came out, I was in the weird place of actually practically using these tools for a long time beforehand. It turns out everybody else who was taking this stuff seriously was computer scientists. So I sort of was there At the early days of like, oh, I know business stuff and entrepreneurship stuff and education, and these things are actually quite u…

AI assessment note: “I just sort of became the go-to person.”

Q You said there's a lot, not a lot of thought going into it. What thought would you like to see going into it? Like, what do you think would be a commensurate level of thought and analysis?

A I think that we need to be built for fast reaction to these models. I, what I'm worried, so there's, uh, Joshua Gans, who's a professor at, uh, University of Toronto. I think It has a really nice model for AI regulation that I think is probably right, which is when you have a new technology, you don't know what the problems and issues are going to be. You do fast follow-up regulation. So you don't try and pre-regulate because you don't know what it's good or bad at, but you do watch what's happening and have rules that you put into place and policies and fast reaction. Now we can talk all about how government's not built to do that, how it's not cooperating well with industry, but I think that's the same way I'd be thinking about open source right now. So we've just released a very powerful model open source. Who is setting up to learn for what the implications of this are going to be? And do they have a pipeline back to the open source makers of these models? Is there something that would stop meta? Is there an event that would stop meta from outsourcing and from open sourcing its models? I don't know who's watching that stuff. Are we have, is there any kind of monitoring system out there to find out how this is disrupting the world one way or another? There doesn't seem to be. So to me, a really responsible view would be sure. Let's release open source. But then let's be watc…

AI assessment note: “a really responsible view would be sure. Let's release open source. But then let's be watching”

Q Why do you think we figure stuff out at university when we want to cut the time to do a assignment, coursework, whatever, but we don't cut the time at work when we still have technically assignments and coursework in our jobs?

A First of all, there's a lot of communication in universities that there isn't elsewhere. When I talk to large companies, no one talks to other people at other organizations very much, right? So you have to hope you hear this from a friend. There's not, but in universities, everyone is like talking about their work and they're all in classes together. And we know that, like, that barrier to spreading information, it's not technical, non-technical, it's just sort of like, oh, cool, my friend showed me that you could solve, you know, it does all the math problems for me, right? Like that, you know, that's one thing, right? I think also, for better or for worse, chatbots are just really good at homework, right? Like, so there's, like, marketing requires you to be an expert in marketing. Chatbots right now, like, if you want to see the future where it's better than human, it's actually in homework, where in most cases it kind of solves the issue, writes an essay better than most people. But it's like, for marketing, you do need to, you know, Work with it a little to be in your house style, to have a conversation with it, to have it understand your context. So there is a little bit more friction there than, um, you know, write a five paragraph essay about George Washington.

AI assessment note: “First of all, there's a lot of communication in universities that there isn't elsewhere.”

Q like Klarna had like, 70% improvements with AI in terms of customer service and cut so many of their workforce. You're seeing especially customer service be the core kind of Trojan horse. Which is replacing, in most cases that I see, 90% now of customer service teams. To what extent do you think I'm over worrying and actually will see the continuing redistribution of talent, not the removal of talent?

A So this is a case where I think we're being a little sanguine about this. I mean, I, I think, you know, again, every technological revolution, um, people lose jobs and then new jobs are created, right? But there's, you know, and we talk about this all the time. There are two big caveats to that. Caveat number one is not always. When the telephone switchboards went from sort of manual to digital in the starting, or not digital at that point, but mechanical in the 19, starting in the 19 thirties, At that point, I think one out of every 16 women had spent time as a telephone operator. Um, it was like a job. And then if you got fired from that, if you were young, you found other jobs. If you were older, you never found another job as good, right? Because you were really good at telephone as a telephone operator. So not every job ends up with a new category replacing it. And the other thing is living through the industrial revolution kind of sucks, right? Like you can look backwards and say, you know, like, oh, great. Everyone got better jobs. They were much richer now. But there were also people, you know, smashing machines because they didn't want their jobs replaced. There was a lot of unrest. There's a reason why there was, that's when the great debates between capitalism and, and Marxism arose because there was unrest during this period that was serious. So I think part of what…

AI assessment note: “this is a case where I think we're being a little sanguine about this”

Q heuristic. Ethan, I was like, Ooh, um, tell me, you said there about kind of the importance of being opinionated and for startups to have strong opinions about where, you know, AGI will be, how they fit into it. Organizational design. If I were to ask you and flip that on you, where are you most opinionated in your views around it? Where would you suggest or point to first?

A So I think education is a good starting point. We could talk about entrepreneurship and other areas, but in education, tutoring is the gold standard for, for interventions based on the research we have. And AI is an incredible one-on-one tutor. Like it's transformative. So But the, but when I find Silicon Valley people and, you know, and, um, and, and AI and education, people often think is like, well, once we have a really good tutor, we don't need teachers or like, I hated this subject in school or people will be self-motivated to learn. Absolutely untrue. People are not self-motivated to learn. Like, and even all the computer scientists out there were like, I was like, yeah, you're an autodidact in some narrow area, but you would have learned nothing about very important topics because you only cared about one topic, right? Like, People need extrinsic motivation to learn. It turns out that there's value in having an instructor guiding the direction of a class, that there's value in putting things into practice. So even an incredible AI tutor that knows you and loves you really well, doesn't sub in for teachers. And also forget all of that. Let's talk about systems. Schools are in a complex system of society and where they are for, you know, providing daycare services to how they fit into education networks, how we do credentialing to The teacher unions too, like there's a bi…

AI assessment note: “So I think education is a good starting point.”

Q I agree. What floors me is the lack of human descriptions around how to use these tools effectively. It's like no one's written, uh, using LLMs for dummies, using AI for dummies, which everyone needs. Why are these providers not doing what is so obviously required?

A I think that if you talk to Silicon Valley people, they are very obsessed with the race for super intelligence. And I totally get it, right? If you could build a machine God, you win. So that's kind of the secret story behind what's going on here is there's a real belief that if scale solves everything, the only, the biggest thing you could do to waste your time is do anything that isn't scaling. Your smartest people have to be scaling. All of your compute has to be scaling and the bigger models will solve all problems. As you were saying, you know, that's a sort of view in Silicon Valley. So they're going to come back and figure this out later because why would you bother? You know, and there's some truth to that, right? I spoke to a very large financial institution, spent a huge amount of money building a GPT three powered sales assistant tool that as soon as chat GVD came out was instantly obsolete. Right. So like, you know, why, why bother with this? I mean, it was a smart idea at the time. They were way ahead of the curve. Right. But I think that the real issue is, is that as a result, All use of this stuff has kind of been dropped, right? There is no manual out there for this stuff. There's not even a dissent, like there's not even a set of points about what the AI is good at and what it's bad at. And as a result, like it's, I would call it documentation by rumor. Like it…

AI assessment note: “they are very obsessed with the race for super intelligence”

Q What's the most concerning future that AI could bring?

A The most concerning future I think is, uh, is one where we lose agency. And not necessarily to the AI systems, but to the systems that incorporate AI. What I mean by that is we have a chance to make AI be used for human thriving. That's not an automatic process, right? That means not firing people when you have AI in your company, but it means figuring out other uses for them that are valuable. It means building systems that help people feel like they're accomplishing more as a result of using these things. And I worry we're not seeing enough people modeling that kind of behavior, that it's all about just the technology itself and then how do we get cost savings.

AI assessment note: “The most concerning future I think is, uh, is one where we lose agency.”

Q I totally get you, and you mentioned there about topping out. Before we discuss kind of the potential topping out and what happens when that does, I do just want to start on actually the four potential outcomes. You highlight this in your book, which I loved, and I just thought it'd be helpful to start there as a framing. What are those four potential outcomes first?

A Of the four outcomes, Twitter, it only talks about like, and you know, the, Press only talks really about one in four. So let me go through one in four first and I'll give you the boring middle, right? So option one is, uh, this is it like the, you know, the models don't get much better or, you know, and it's sort of this whole thing sort of fizzles out. I think this is unlikely because I think not only will models get better, but also we haven't even started integrating them into work yet, right? Like the way you work with these things is an insane process of actually using a chat bot and having a conversation with chat bot is how people are using it for work at this stage. So We're, we're not even at the stage of integrating, but it's possible that things kind of, you know, in this, in which case we have 10 years or so of integrating the system solely into human systems. I think everything sort of stabilizes out where it is. We're not going to see, we'll, we'll see an economic improvement, but we probably don't see kind of a massive large scale shift, except some industries change more than others, right? I think, I think, you know, it's very likely that photography changes a lot and that there's other fields and customer service changes a lot with our current systems, but that's one option, right? Nothing much happens. Then option four is the machine God, right? Um, we sort …

AI assessment note: “So let me go through one in four first and I'll give you the boring middle”

Q You said there's a lot, not a lot of thought going into it. What thought would you like to see going into it? Like, what do you think would be a commensurate level of thought and analysis?

A I think that we need to be built for fast reaction to these models. I, what I'm worried, so there's, uh, Joshua Gans, who's a professor at, uh, University of Toronto. I think It has a really nice model for AI regulation that I think is probably right, which is when you have a new technology, you don't know what the problems and issues are going to be. You do fast follow-up regulation. So you don't try and pre-regulate because you don't know what it's good or bad at, but you do watch what's happening and have rules that you put into place and policies and fast reaction. Now we can talk all about how government's not built to do that, how it's not cooperating well with industry, but I think that's the same way I'd be thinking about open source right now. So we've just released a very powerful model open source. Who is setting up to learn for what the implications of this are going to be? And do they have a pipeline back to the open source makers of these models? Is there something that would stop meta? Is there an event that would stop meta from outsourcing and from open sourcing its models? I don't know who's watching that stuff. Are we have, is there any kind of monitoring system out there to find out how this is disrupting the world one way or another? There doesn't seem to be. So to me, a really responsible view would be sure. Let's release open source. But then let's be watc…

AI assessment note: “let's be watching over the next six months to get a sense of what this is good”

Q I'm so sorry to be so naive. Why, if Code Interpreter is such a generational defining product for analysts, why would they walk away from it or not walk away from it, but, you know, not progress in the same manner as they started?

A OpenAI abandons products like crazy. I think these products are passion projects from various people. Again, they want to build the machine God. If you have any talented people, you're going to have them doing, you know, building the next technology for AGI. And if you have staff, that's what you throw it at. If you have compute, that's what you throw it at. I mean, they're incidentally making three billion dollar run rate this year, I think by like just accident, but there isn't like a, there isn't really a product there right now. It's, it's the chat bot and the API and the system gets smarter to solve more problems. I think a lot of people in this space are just assuming scale solves issues. So why would I buy or development solves issues? So why would I bother spending some time thinking about, you know, how to productize this when the proxy be obsolete in a year anyway?

AI assessment note: “they want to build the machine God. If you have any talented people”

Q like Klarna had like, 70% improvements with AI in terms of customer service and cut so many of their workforce. You're seeing especially customer service be the core kind of Trojan horse. Which is replacing, in most cases that I see, 90% now of customer service teams. To what extent do you think I'm over worrying and actually will see the continuing redistribution of talent, not the removal of talent?

A So this is a case where I think we're being a little sanguine about this. I mean, I, I think, you know, again, every technological revolution, um, people lose jobs and then new jobs are created, right? But there's, you know, and we talk about this all the time. There are two big caveats to that. Caveat number one is not always. When the telephone switchboards went from sort of manual to digital in the starting, or not digital at that point, but mechanical in the 19, starting in the 19 thirties, At that point, I think one out of every 16 women had spent time as a telephone operator. Um, it was like a job. And then if you got fired from that, if you were young, you found other jobs. If you were older, you never found another job as good, right? Because you were really good at telephone as a telephone operator. So not every job ends up with a new category replacing it. And the other thing is living through the industrial revolution kind of sucks, right? Like you can look backwards and say, you know, like, oh, great. Everyone got better jobs. They were much richer now. But there were also people, you know, smashing machines because they didn't want their jobs replaced. There was a lot of unrest. There's a reason why there was, that's when the great debates between capitalism and, and Marxism arose because there was unrest during this period that was serious. So I think part of what…

AI assessment note: “this is a case where I think we're being a little sanguine about this.”

Q Why do you think we figure stuff out at university when we want to cut the time to do a assignment, coursework, whatever, but we don't cut the time at work when we still have technically assignments and coursework in our jobs?

A First of all, there's a lot of communication in universities that there isn't elsewhere. When I talk to large companies, no one talks to other people at other organizations very much, right? So you have to hope you hear this from a friend. There's not, but in universities, everyone is like talking about their work and they're all in classes together. And we know that, like, that barrier to spreading information, it's not technical, non-technical, it's just sort of like, oh, cool, my friend showed me that you could solve, you know, it does all the math problems for me, right? Like that, you know, that's one thing, right? I think also, for better or for worse, chatbots are just really good at homework, right? Like, so there's, like, marketing requires you to be an expert in marketing. Chatbots right now, like, if you want to see the future where it's better than human, it's actually in homework, where in most cases it kind of solves the issue, writes an essay better than most people. But it's like, for marketing, you do need to, you know, Work with it a little to be in your house style, to have a conversation with it, to have it understand your context. So there is a little bit more friction there than, um, you know, write a five paragraph essay about George Washington.

AI assessment note: “First of all, there's a lot of communication in universities that there isn't elsewhere.”

Q I totally agree and get you. What should I, and what should my fellow venture investors change then about the way that we invest, do you think?

A What you should be thinking about is have a position on the future, and the startups you talk to have to have a position on the future of AI. How good does it get, and how does your model work? The second thing I think people need to be thinking about is how actual adoption happens again, right? It used to be that if you have a large enough market to play with, we just go after all of it, and, you know, some part of it starts to respond, and we double down on that section. You're gonna be much more opinionated about how you imagine your technology being spread or adopted. How does it spread throughout our organization? Is it How does it fit? Um, you know, I, I think that there's just, it's, it's just requires people to have more plan and strategy than they did before, rather than just letting the market tell them the answer.

AI assessment note: “What you should be thinking about is have a position on the future”

Q heuristic. Ethan, I was like, Ooh, um, tell me, you said there about kind of the importance of being opinionated and for startups to have strong opinions about where, you know, AGI will be, how they fit into it. Organizational design. If I were to ask you and flip that on you, where are you most opinionated in your views around it? Where would you suggest or point to first?

A So I think education is a good starting point. We could talk about entrepreneurship and other areas, but in education, tutoring is the gold standard for, for interventions based on the research we have. And AI is an incredible one-on-one tutor. Like it's transformative. So But the, but when I find Silicon Valley people and, you know, and, um, and, and AI and education, people often think is like, well, once we have a really good tutor, we don't need teachers or like, I hated this subject in school or people will be self-motivated to learn. Absolutely untrue. People are not self-motivated to learn. Like, and even all the computer scientists out there were like, I was like, yeah, you're an autodidact in some narrow area, but you would have learned nothing about very important topics because you only cared about one topic, right? Like, People need extrinsic motivation to learn. It turns out that there's value in having an instructor guiding the direction of a class, that there's value in putting things into practice. So even an incredible AI tutor that knows you and loves you really well, doesn't sub in for teachers. And also forget all of that. Let's talk about systems. Schools are in a complex system of society and where they are for, you know, providing daycare services to how they fit into education networks, how we do credentialing to The teacher unions too, like there's a bi…

AI assessment note: “So I think education is a good starting point.”

Q Is that really an order of magnitude improvement if we compare that post-classroom? You could give me incredible high-quality videos of you Talking, lecturing, giving examples that you give to your students now, very easy to do, versus that AI tutor. Is it, is it 20% better? Sure, maybe it's personalized, but is it really an order of magnitude better?

A Education is a complex system. So I think order of magnitude is a very weird thing to talk about, because every student has their own talents, abilities, interests, and gaps. The early work on, in one-on-one tutoring, we don't talk about order of magnitude improvement, because that doesn't really work in the education world. It's very hard to say what an order of magnitude is. But we could talk about grades a lot and the classic study that is probably would not be replicable, but it sets up our model is that one-on-one tutoring, um, according, it creates a two sigma increase in, uh, in classroom outcomes. That's two standard deviations, which is, you know, a fairly huge improvement. You go from the 50th percentile to the 97th percentile in class. We have no idea if that's gonna hold up with, you know, AI tutoring, but if we could do that, that is as amazing an improvement as you could possibly ask for. I mean, a 10% improvement is amazing. I, I kind of feel like aiming for order of magnitude education, if we can get improvement in a system, we're in great shape.

AI assessment note: “I think order of magnitude is a very weird thing to talk about”

Q When you look at the pervasiveness of AI and specifically ChatGPT in homework and in coursework and in the answers that many students give today, is there any point in university or educational facilities doing homework or coursework when it's largely done by AI today?

A Of course there is. We, like, look, everybody was already cheating. Like, there's this great study at a repeating university that found that homework It improved, when you did the homework, it improved something like 80% of people's test scores in 2008. And by 2020, it only helped 20% of people. And that's not because homework stopped helping. It's because everyone was cheating. And so we have ways around this. There's really two options in how to use AI in education. One of them is to ban it cautiously, right? People are still going to use this explainers and stuff like that, but you, you have in-class tests and blue book writing, like we've solved this problem in math. And like you make people do exercises and do work. Nobody likes it, but there's no shortcut to learning. It sounds dumb. It's like what your teacher said. Oh, it turns out it's true. You need to do a grinding amount of work to understand something. You need to do interleaved practice. You need to like, there's a lot of stuff you need to do to learn something. And so we absolutely can make you do blue book work in class. We absolutely can install, um, terrible monitoring systems. I don't like this approach, but like a couple of companies already have this at watch what you're typing and make sure you're not pasting stuff in from AI. Again, I don't necessarily recommend it, but like, These are possibilities. Like…

AI assessment note: “Of course there is. We, like, look, everybody was already cheating.”

Q I agree. What floors me is the lack of human descriptions around how to use these tools effectively. It's like no one's written, uh, using LLMs for dummies, using AI for dummies, which everyone needs. Why are these providers not doing what is so obviously required?

A I think that if you talk to Silicon Valley people, they are very obsessed with the race for super intelligence. And I totally get it, right? If you could build a machine God, you win. So that's kind of the secret story behind what's going on here is there's a real belief that if scale solves everything, the only, the biggest thing you could do to waste your time is do anything that isn't scaling. Your smartest people have to be scaling. All of your compute has to be scaling and the bigger models will solve all problems. As you were saying, you know, that's a sort of view in Silicon Valley. So they're going to come back and figure this out later because why would you bother? You know, and there's some truth to that, right? I spoke to a very large financial institution, spent a huge amount of money building a GPT three powered sales assistant tool that as soon as chat GVD came out was instantly obsolete. Right. So like, you know, why, why bother with this? I mean, it was a smart idea at the time. They were way ahead of the curve. Right. But I think that the real issue is, is that as a result, All use of this stuff has kind of been dropped, right? There is no manual out there for this stuff. There's not even a dissent, like there's not even a set of points about what the AI is good at and what it's bad at. And as a result, like it's, I would call it documentation by rumor. Like it…

AI assessment note: “they're going to come back and figure this out later because why would you bother?”

Q Final one for you, Ethan. What question are you not ever asked that you think you should be asked more?

A The question that I think people should be asking and that I don't have an answer to yet is like, why are so many people bouncing off these systems? Like, why are they not, you know, why are so many people using them a little bit and not forever? Because I don't think it's just simple. Like it didn't work. People are getting kind of freaked out by these things in ways that we don't really understand how humans are relating to these tools. Like, we always talk about the systems, the technology, the, you know, what, how industry is gonna change, and I don't mean just like the dating thing, which tends to be like, well, you have a relationship with AI, but like, how are we relating to these kind of tools? Um, that's one of the questions, and let me do a second take on this, um, a second one. I, I think a lot about, people don't ask me enough about meaning of work. That matters a lot, right? Um, you know, Graeber's bullshit jobs, I think, was mostly not correct based on survey data and other stuff we saw, but it's real. People do feel alienated from work. People do, like, Most employees say they're bored at least 10% of the time at work, but they're doing work that they feel is meaningful. When you survey people, most people think their jobs matter in the world. What's going to happen that I'm very worried about is when you realize as a middle manager that AI does your work and nob…

AI assessment note: “The question that I think people should be asking”

Q By the way, your Twitter game is fantastic. So like, don't change that at all. I love it. Um, I want to start though, and it's pretty perfect timing. I said we were pretty casual in how we did this. You know, we saw the new Llama 3.1 model come out yesterday. I'm just really intrigued to hear your thoughts, Ethan. What did you think? Is it what you expected?

A There's like four or five dimensions that the Llama model is super interesting in. Um, we could talk about open weights and open source being one model. I, I'm not surprised that they caught up with, uh, the leading edge state of the art models. I think people are probably over, uh, underestimating how much ammunition the Closed source labs have and are going to release in the near future. Um, but I think it's great. We now have an open source GPT four capable model and it's going to be everywhere. And you know, it's just interesting because how much of that gap gets closed by that model. So, you know, every national government had worse AI than any, than, than every kid in Mozambique had access to through GPT four. Oh, so now there's a openly available fine tune model. We're going to see a lot of weird effects from AI that were delayed happen as a result, actually using it. It's, it's, Pretty good. I mean, I, I don't think it stands out compared to a Claude at this point or something else, but it will soon because people will be working on it, and it is a downloadable open, open weights model, which is kind of a big deal.

AI assessment note: “I'm not surprised that they caught up with, uh, the leading edge”

Q I have to, you said so many great things there. Would you tell your students today that they have to move to the Valley if they want to increase their chances of winning?

A That's an empirical result from a bunch of studies. Companies that, you know, there's been a study of Israeli companies in the Valley, New York companies. Like, it's just, the issue is, is that that's where the connections are, and it turns out Zoom only gets you so far. I mean, the average distance, at least pre-pandemic, I would, I'd be surprised if it actually changed. Um, the average distance between a VC and a company to invest in is about 40 miles. Like, that's because when you look at what, where VCs spend their time, it's networking and it's monitoring. It's networking with other, with, you know, and learning about companies, and then it's monitoring the portfolio companies. And that's much easier when you're local. Zoom doesn't let you do monitoring the same way. In fact, when a direct flight is added between SFO and another city, the VC investment in that city goes up. Because it's just getting, it's easier to fly there and help do, ah, and do monitoring there. It's a local business, right? Everyone's like, oh, it's global. It's connected. It's a local business.

AI assessment note: “That's an empirical result from a bunch of studies.”

Q Can I dive into a couple of the different market participants? We've already touched on some of them, but I want to start on AI labs. We've mentioned Lama and model progression earlier in terms of the AI labs. What do the big AI labs not understand about companies themselves? Do you think?

A I think that's such an important question. I mean, there is just the products being released are just super weird. I think there's very little consideration of use cases. I mean, you look at the number of people inside these organizations who've worked at large companies. I, I often joke like, you know, When I go to, when I go to the West Coast, there's like, you know, it's all like cold plunges, and how do you live forever, and really, you know, and then in the East Coast, it's like, we just, you know, we're drinking coffee till we die, and the goal is like, get our work done, get home, like, you know, like, it's just a different, like, in a large company, it's very different, and there's a lot of, like, contempt, I think, for large companies. That's where most smart people are, right? Are in large organizations doing, you know, other work that is not Silicon Valley work. For every coder, there are 16 managers, right? And, you know, I, I think that there's not a sense of what this stuff does for them, and as a result, there's a lot of half-built products that are brilliant and then get walked away from. Code Interpreter is, is a huge world-changing product for data analysts that got partially abandoned by OpenAI. They haven't, they haven't moved the needle in that sense. Chatbots and APIs remain the main area. Almost all the documentation is technical documentation, and almost…

AI assessment note: “there's very little consideration of use cases.”

Q look like when kids come home from school, they just have perplexity or open up and they have another tutor with them where, as you said, in most cases, they end up doing the work for them. And so they don't learn. Is it a crutch that, and they, I don't understand actually intangible reality What does the future of education look like with AI, and why is it optimistic?

A First of all, there's a couple things you need to know about learning that people don't tend to think about, which is learning is hard and sucks, and that what makes you feel like you're learning isn't what's learning. Like, you have to do grinding work. There's no solution to it. It's just like any other thing, like exercise or anything else. You have to be pushed to desirable difficulties where you're having trouble if you're not failing at a thing, you're not working hard enough. Like, that's, which is why you often need intrinsic motivation. And the second thing is, We actually have some research. We know things like active learning where you're in a classroom doing activities, um, beats the idea of passive, just receiving a lecture. When we have those sets of pieces, there's been a move that kind of fizzled called flipped classrooms that has some early evidence in its favor, which suggests this idea of like classroom should be about doing stuff and outside of class should be about getting the basics, right? Because we can get you to do stuff in, in the classroom setting. So That would mean that outside of class that what that practically meant is you watch videos outside of class, your teacher talking. So the lecture stuff is all outside of class. That's your homework. Read the book, do that. Then your homework is in class where you can mess up in front of people and work …

AI assessment note: “instead of having a passive video, you watch, you'll have an AI tutor outside of class.”

Q give you another one, uh, which is, uh, compute is the currency of the future, what Sam Altman said, and energy is a concern, When looking at the energy requirements that this next generation of AI will bring, how do you think about the energy requirements required for this next generation of AI usage in society, and whether Sam is right that, you know, computers the currency of the future?

A Sam believes in AGI, and he believes that it's going to be achievable in the near term, right? And when you talk to open AI insiders, they feel the same way. If that's the case, if, if intelligence on demand is the case, intelligence on demand is power hungry, and there's infinite demand For intelligence on demand. Cause there will be, right? Like if you have an AGI, I want that to be looking over all of my medical records and monitoring our airspace and, you know, finding scientific ideas and helping me with a project I have to do and also booking tickets for the ultimate trip. Like there is infinite demand for intelligence, right? So then compute becomes the currency and energy becomes the big deal. And we're going to build a lot of nuclear power plants, I guess, and, you know, in relatively short order. Um, And, you know, it seems like that's a pro or AGI figures how to do fusion, and it doesn't matter, or we all get turned into batteries all in matrix, although we don't produce enough wattage, um, you know, so I don't think that's really the issue of training data. That's what the AIs will use us for. Um, but anyway, the, um, mostly joking. Right now, I think the energy debate's an interesting one, because again, it's one where doomers and, you know, and, uh, and optimists sort of like to talk about, because on the downside risk, when I meet people who are skeptical at AI, …

AI assessment note: “So then compute becomes the currency and energy becomes the big deal.”

Q By the way, your Twitter game is fantastic. So like, don't change that at all. I love it. Um, I want to start though, and it's pretty perfect timing. I said we were pretty casual in how we did this. You know, we saw the new Llama 3.1 model come out yesterday. I'm just really intrigued to hear your thoughts, Ethan. What did you think? Is it what you expected?

A There's like four or five dimensions that the Llama model is super interesting in. Um, we could talk about open weights and open source being one model. I, I'm not surprised that they caught up with, uh, the leading edge state of the art models. I think people are probably over, uh, underestimating how much ammunition the Closed source labs have and are going to release in the near future. Um, but I think it's great. We now have an open source GPT four capable model and it's going to be everywhere. And you know, it's just interesting because how much of that gap gets closed by that model. So, you know, every national government had worse AI than any, than, than every kid in Mozambique had access to through GPT four. Oh, so now there's a openly available fine tune model. We're going to see a lot of weird effects from AI that were delayed happen as a result, actually using it. It's, it's, Pretty good. I mean, I, I don't think it stands out compared to a Claude at this point or something else, but it will soon because people will be working on it, and it is a downloadable open, open weights model, which is kind of a big deal.

AI assessment note: “I'm not surprised that they caught up with, uh, the leading edge”

Q Can I ask you on the flip side, we have the companies themselves. What are companies getting wrong about AI that they should know more about?

A I mean, I speak to organizations all the time. I mean, first of all, just from a perspective, almost nobody uses these systems. I mean, they all tried chat TPT, right? Every, when I asked my hand, everybody's tried chat TPT, almost always the three, five version of before about Five to 10% of people in any room, whether, by the way, Silicon Valley, actual people, right, who aren't at a lab, whether that's at a large bank, whether that's at a conference of innovation professionals, maybe five to 10% have used those models, and maybe two or three percent have used 10 hours, which has been my, you know, sort of guideline, you know, minimum number. And I think, again, there's no onboarding. You're faced with a chatbot, and when people are faced with the tyranny of the blank page, They panic. What do you talk to the system about? Right? And like, there's no information. There's no instructions. And so people aren't really using it. So the issue is that partially it's that they need to adopt because when people start using it, they find uses, right? So a new study just came out of Denmark of people who are using chat GPT and, you know, in knowledge intensive work environments. And, you know, they're estimating that in, you know, over 30% of their tasks, they're saving 50% of their time. So once people use it, they find productive uses. So then the question becomes, How are you harnes…

AI assessment note: “first of all, just from a perspective, almost nobody uses these systems.”

Q look like when kids come home from school, they just have perplexity or open up and they have another tutor with them where, as you said, in most cases, they end up doing the work for them. And so they don't learn. Is it a crutch that, and they, I don't understand actually intangible reality What does the future of education look like with AI, and why is it optimistic?

A First of all, there's a couple things you need to know about learning that people don't tend to think about, which is learning is hard and sucks, and that what makes you feel like you're learning isn't what's learning. Like, you have to do grinding work. There's no solution to it. It's just like any other thing, like exercise or anything else. You have to be pushed to desirable difficulties where you're having trouble if you're not failing at a thing, you're not working hard enough. Like, that's, which is why you often need intrinsic motivation. And the second thing is, We actually have some research. We know things like active learning where you're in a classroom doing activities, um, beats the idea of passive, just receiving a lecture. When we have those sets of pieces, there's been a move that kind of fizzled called flipped classrooms that has some early evidence in its favor, which suggests this idea of like classroom should be about doing stuff and outside of class should be about getting the basics, right? Because we can get you to do stuff in, in the classroom setting. So That would mean that outside of class that what that practically meant is you watch videos outside of class, your teacher talking. So the lecture stuff is all outside of class. That's your homework. Read the book, do that. Then your homework is in class where you can mess up in front of people and work …

AI assessment note: “I think flipped classrooms are a very natural fit... with AI based approaches.”

Q I totally agree and get you. What should I, and what should my fellow venture investors change then about the way that we invest, do you think?

A What you should be thinking about is have a position on the future, and the startups you talk to have to have a position on the future of AI. How good does it get, and how does your model work? The second thing I think people need to be thinking about is how actual adoption happens again, right? It used to be that if you have a large enough market to play with, we just go after all of it, and, you know, some part of it starts to respond, and we double down on that section. You're gonna be much more opinionated about how you imagine your technology being spread or adopted. How does it spread throughout our organization? Is it How does it fit? Um, you know, I, I think that there's just, it's, it's just requires people to have more plan and strategy than they did before, rather than just letting the market tell them the answer.

AI assessment note: “requires people to have more plan and strategy than they did before”