The Exchanges

Q Could we actually get into that? I'd love to hear sort of what you view as the consensus definition of AGI today. What's wrong about it? And then what do you think is the right way to measure or calibrate against that?

A Yeah. The sort of consensus definition that I think is most popular in sort of the AI industry right now is that AGI is a system that can do like the majority of economically useful work that humans can do. I, I think Vinod, uh, gets credit for joining this one. And, um, I, you know, I think it's a useful definition actually, uh, you know, look, I spend my day job building application and there is legitimate economic value that is sort of unlocked by the current regime with language models. Um, however, I don't think it's a good EGI definition though. Um, you know, I think it's a good definition of systems that are useful and economically useful, but, you know, I kind of joke that like, I think it says more about what many humans do for work than it does about actual general intelligence. And, uh, Francois definition, which is the one that I think is the right one is, uh, this definition that general intelligence is a system that can effectively, efficiently acquire new skill. That's, that's it efficiently acquiring new skill and being able to solve these open-ended problems with that ability. And here's sort of the simple, like maybe, um, argument in this line of thinking is, you know, we've had AI systems over the last 1015 years that can now, uh, you know, win at poker, uh, fold proteins, drive cars, win at chess. And yet I can't take any system that was like trained to beat…

AI assessment note: “The sort of consensus definition... is that AGI is a system that can do”

Q Is there anything you can share in terms of Um, adoption or metrics or usage by Zapier users or customers of, of your AI products?

A Yeah, we've got, um, at this point, over fifty million AI tasks have run on the platform to date over the last year and a half or so since we started tracking. So this is like, you know, think of a Zap, right? Where it's like you've got a trigger instead of actions where one of those actions is an AI step. Dominantly, this is open AI or a chat to PT step. Where, you know, users doing content generation or feature extraction or summarization, um, using AI in the middle of a workflow, uh, is, is kind of the dominant way people are adopting AI today. Um, over the last couple of months, we've introduced, uh, uh, other products in our AI space. So we're using AI basically across the entire product. We've, we launched a new product called Zapier, um, central, which are effectively these AI bots that, um, you don't have to build effectively. Uh, you know, the classic way I think most people experience Zapier is you have to build In the editor, right? You go have to, you know, do lots of configuration and click, click, click in order to get your zap set up and just tuned to the way you want. And one of the cool things with these new AI bots is you program the natural language. And we're not actually even doing natural language to structure mapping. It is a pure inference based engine, interpreting the user's instructions of what they want the bot to do and getting access to the, all th…

AI assessment note: “over fifty million AI tasks have run on the platform to date”

Q value is the fact that it is generalizable in different ways that didn't exist before. And it does open up the aperture in terms of one system that's kind of trained broadly, but then can do a lot of very specific subtasks. So could you explain more about why you don't feel that just scalability of LLM sort of leads in this direction eventually or scalability of some multimodal model?

A You know, the, the, the sort of claim goes like this, uh, effectively what large language models do today is they are high dimensional memorization systems, right? They are trained on lots of training data. They're able to find and generalize patterns off of the training data that they're trained on and then apply those in, in new contexts. And memorization is a form of intelligence, I would claim. Um, but it's not a form of general intelligence, right? We need something. There's something more that we need in order to be able to go discover and invent alongside us. You know, this is the things that I care about, like with AGI. This is why I want to build AGI. I think like, if we want to pull forward the future and actually have AI systems that are able to, you know, discover new branches of physics or pull forward our understanding of the universe, um, pull forward like new therapeutics. The answers to those don't show up in high dimensional patterns from our existing training data, because. Like the answer is, is literally unknown, right? The pattern is unknown. In fact, you might be able to find some sub patterns that can apply in like similar reasoning chains. And that's actually how current sort of AI agent systems work, right? If the reasoning chain that you need an agent to follow is simple enough such that the reasoning chain shows up in an abstract way in the training …

AI assessment note: “effectively what large language models do today is they are high dimensional memorization systems”

Q you know, you've, you've now established this arc prize, which I think is super exciting. It's a million dollar prize towards, um, you know, an open source model that, you know, meets certain criteria against your metrics of artificial general intelligence. Why do it as a prize versus investing in companies or, you know, taking a funding, uh, more traditional funding of startups or efforts model versus a prize model?

A I think outsiders are needed. Um, you know, there, there was 300 teams that actually competed in the ARC, like small version of the contest last year in 2023. And if you go look at all the teams that competed, you know, these are like one or two person teams. They are outsiders to the industry. They're not working in AI startups. Many of them don't even live in like the Bay area or Silicon Valley or California. It's a very globally distributed set of people with new ideas that are working on this stuff. I am more confident actually that, uh, or I guess I would bet that the solution arc probably comes from an outsider. Um, I think it's probably gonna come from somebody who's sort of not indoctrinated in the current way of thinking about language models and scale. Arguably like the solution arc doesn't even require that much scale. Um, you know, the, the cool thing about the puzzle, the RQGI values, it, it, it's like kind of a minimal reproduction of general intelligence. Uh, it fits into a two by two game board. That's like at max, like 15 by 15 squares big. Like it's, it's so small and reproducible. The data fits into such a small, uh, small set that, um, it's quite likely actually that the solution, um, it, it can be like written in like 10,000 lines of code or less. Uh, and it's not gonna require these like, you know, gigantic You know, two hundred billion large parameter mod…

AI assessment note: “I think outsiders are needed.”

Q So open source software as well as just open source ideas, papers, data sets, et cetera, have really helped drive multiple areas of science and technology forward. How do you think about open source software in the context of AI in particular, given some of the regulatory and other movements that have been happening at both the California level, the national level, et cetera?

A My, my beliefs here are formed through how much we stall that I think on AGI progress. We still need fundamental research breakthroughs. We still need to be fundamental new ideas. And I think the internet and open source has been one of the world's best inventions, uh, in order to generate new ideas. Um, and so I think if you care about actually discovering AGI in our lifetime, then I think it's sort of incumbent to try and promote. Things that increase the likelihood that we're generating new ideas. Um, and having lots of AI researcher brains or would be AI researcher brains sort of encountering this stuff and it's not locked and closed behind, you know, a hiring process at a big lab. And so, you know, I'm, I'm very much in favor of supporting open progress, open research sharing, especially at the like foundational scientific level, because we, we just need new ideas. Um, and I think the best way to generate those ideas is through open source and open sharing at this point. I mean, the proof here is like literally open AI, right? Like the sort of genesis of the company came out of a published research result. From, from Google. Um, and sadly, I don't think that's likely to happen now as a result of kind of a lot of the commercialization and market incentives, causing a lot of frontier publishing getting, getting closed up because now these, you know, companies sort of have, t…

AI assessment note: “I'm very much in favor of supporting open progress, open research sharing”

Q When you talk about AGI, um, you know, I think there's some books like Blindsight Which tries to differentiate between intelligence and sentience, right? Self-awareness versus actually being able to intelligently do things. When you talk about EGI, is there an embedded concept of sentience in it or is it purely intelligence?

A I'm not a philosopher. So, uh, like I'm probably the worst person to ask about this question. Look, I want to live in the future. That's like kind of one of the things I've always been really excited about. Like, you know, if I can help pull forward the future, I want to. And I think one of the best ways we could pull forward the future is to invent systems that Can invent and discover alongside us. And I think in order to do that, we need this general form of intelligence or a system that can represent this, demonstrate this general form of intelligence about being able to efficiently acquire those new skills and help us solve these open-ended problems. So I, I don't, um, like I haven't thought deeply about like, okay, well, is that system sentient conscious?

AI assessment note: “I haven't thought deeply about like, okay, well, is that system sentient conscious?”