The Exchanges

Q How about naming? Yeah. Do you want to name one?

A Yeah. Let me take an example, our segment anything model, which is a little bit different than our, than our Lama model. Um, but, but I think has been the one that has been just incredibly impactful in terms of people quickly building on it. Our, our segment anything model is one where you take an image and it gives you a detailed segmentation. Of that, uh, of that we released it back in April, including a lot of, um, tools and data to, to go along with it, and, and within days we had people who had built up applications essentially for, um, Conservation applications, so being able to track down some species who may be endangered, using that to, to follow them. We had people use it for the treatment of medical images, so segmenting cells from some, some of these images, and it's been wonderful to see that, that explosion, that explosion of work. Um, on the language side, we also saw many people build up all sorts of, Um, different tools. And in particular, the, the work that we're most excited about is, uh, the work on efficiency, to be honest with you. Um, there's so much that we can do to make these model more compact and, and efficient and, and, and running, um, really, really fast with low energy. And I think that's one of the things that I've been most excited about seeing. There's lots of other applications too.

AI assessment note: “Let me take an example, our segment anything model”

Q Say, say more about it. Talk about what do you think is not being done that could be done?

A I see it every day. Uh, I mean, and I'll, I'll, I'll open up a little window. Like one of the reasons I was super excited about joining Cohere is because it's one of the few places that, you know, that we have a team that does research. So I get to see, you know, day to day what's happening in research. We have a team that does modeling. So I get to see the models that were built and look at the evaluations, a full spread of evaluations, and we have a product. That's product is an agentic platform that is going to real clients. So you get to see the whole thing, and I see something that our models can do, and I see some things that we've built into the products, and then we go, and there's a lot of customers that are not using the full functionality for all sorts of reasons. Um, so I think like that, that between like what we have in terms of capacity versus what's being deployed right now, there's a big gap between that. Sometimes the reasons are, um, are, uh, capacity questions. Like a lot of actual, we talk a lot about super intelligence, big models. In reality, paying customers want like a good trade-off in terms of performance for efficiency. So, you know, we'll train bigger models, but we'll deploy smaller models because it gives us that trade-off. It's like good enough intelligence to get the job done. And I'm like, well, we could give you so much more. No, no, no, it's …

AI assessment note: “between like what we have in terms of capacity versus what's being deployed right now”

Q view that in order to build the products that we want to build, we need to build for general intelligence. Now we talked a little bit about why that is now relevant, the path towards general intelligence, but now I'm kind of left with another question, which is why does Meta need to build general intelligence in order to build the products that you want to build? I mean, yeah.

A I mean, just looking at like a couple of the AI products we've, we've released this year, you know, one of them is the meta AI assistant. People who are in the US have been able to try this out on some of our platforms where you can essentially ask for questions and ask for assistance. In that case, you know, there's a sense that, that it has to understand a very large spectrum of information to, to be able to, to do well. Um, and As we incorporate more data and as we perfect this, this instant, the more it's going to have essentially world knowledge, the better it's going to be. Um, another example is, um, for those who've been following our work on, on, uh, our devices, the, the smart glasses that we released, uh, earlier this year also come now, uh, with an AI model also accessible mostly in the US at this time. Um, there too, you know, you have essentially a more, uh, Embodied version of this meta AI assistant that sees the world as you see it, that is able to take on some action. In this case, the actions are not just words. It can take pictures. It can provide information. It can record information. Um, and so to be able to do well in a wide set of different tasks with a wide set of different people, different environments. You need to have to move towards more general intelligence. Um, that's really the, where that, where that connects, you know, the research work we're …

AI assessment note: “to be able to do well in a wide set of different tasks... You need to have to move towards more general intelligence.”

Q web and they can find something in a session, but as soon as you close that session, they forget it. Um, and I guess the reason why I'm going there is because a way that some people have suggested solving both of these is just making the context window massive and then just becoming efficient in the way that you navigate that. Uh, what do you think about that hypothesis?

A Um, the, the, the two concepts are related, but they're not exactly the same. Um, and so memory is, is really about how do you address sort of what information to pull in, in the context of the task you're trying to solve. Continual learning makes the assumption that the context keep on changing. Therefore, what you've learned keeps on changing. So there's a notion of non-stationarity that is really key to continual learning. I confess I have a little bit of trouble with continual learning as a concept because I feel the community has never been able to nail, like, how do we articulate the problem in a way that we all agree on it? And so everyone who does work on continual learning takes a different flavor of it, which makes it at least in my eyes, and I haven't worked a lot in this area, but makes it a little bit hard to know whether we're making progress or not. On, on, on memory, it's a little bit more standardized. The tension really is about, it's a question of efficiency and relevance. So the way to measure whether you're doing that is a little bit better standardized and you don't want to be just sort of remembering everything. Um, and so it's a little bit better standardized how we articulate the tasks.

AI assessment note: “the two concepts are related, but they're not exactly the same.”

Q Right. Uh, Joel, I'm curious what you think. How do we know that humans are conscious? Like, isn't that a place we want to start with?

A I think, I mean, and Anil alluded to that, you know, the lack of a crisp definition is, is one thing that would be necessary, at least, you know, and I want to get back to your point, right? There is a spectrum to talk about these ideas. On this panel, you have people that are mostly coming from a scientific engineering discipline. And so, so from that point of view, and certainly for myself, I like when I have a crisp definition, that's a verifiable, testable, you know, definitions from which I can build hypothesis that are falsifiable and so on. That is, The types of methods that I apply to this, this particular study. So for me, having a crisper definition of consciousness, even in humans, and you alluded to animals and other forms of life, would be very useful to help us progress in that conversation. In absence of that, It's all quite esoteric, which again, you know, there, there's an opportunity to, to also bring in lots of other disciplines to talk about it, whether it's literature, social sciences, and so on, to, to give points of view on that. But as long as you're treating it from a, from a scientific investigation point of view, having this crisp notion is really helpful to making, to making progress.

AI assessment note: “the lack of a crisp definition is, is one thing that would be necessary”