The Wisdom Wall

“My worry about closing the doors down now is that the models are only getting better. And so if we don't release them now, we really miss an opportunity to develop the muscle we need to make these models safer. Um, and I don't think today's model are the ones that are gonna, you know, bring to the front the hardest…”

“That's the part that the reasoning models don't do. They do really well at like one level of granularity... But the going back and forth between different levels of sort of resolution of action, it's really hard. So on the technical terms, we call it hierarchical planning. That's really hard to do that decomposition…”

“I, I do believe they can generalize. I think they generalize relatively narrowly, or at least, you know, as long as you stay close, you get a good manifold of information. When you start to go really far afield from your data, because the dimensions are so large, you get all sorts of, all sorts of noise.”

“a lot of the ingredients of how to do reasoning have been explored in AI for 40 years. Um, And, and are published and well known to, to anyone who's taken even an undergrad level course in AI. Um, so I'm not saying they, that there's not any innovation in the work that, that open AI is doing or in the work that's…”

“World models are absolutely essential when you want to build agents, because these agents are going to take actions, which is going to change the world. You want to be able to predict these effects.”

“And that's why, honestly, for many years I've been so much an advocate for open science. I just don't believe that you can keep these ideas boxed in unless you're willing to keep people boxed in, which we are not willing to do. Um, and so I don't, I don't, I don't think we have a way to close the ideas. We should…”

“extending the context length is kind of the easiest way to go about it, but there's quite a bit of progress that is, that is being made on this.”

“In reality, paying customers want like a good trade-off in terms of performance for efficiency. So, you know, we'll train bigger models, but we'll deploy smaller models because it gives us that trade-off. It's like good enough intelligence to get the job done.”

“It's also the people who are in leadership position, which instead of like writing out a memo suddenly can go out and like produce a full fledged prototype. They don't need, you know, 10 people, 10 staffers to help them produce their prototype. They have an idea, they can quickly prototype it.”

“those who are able to prototype in this way, it doesn't mean that whatever your vibe coded into a weekend suddenly turns into a hundred million dollar business, right? But it's a way to communicate with your teams, your intention. So as long as you have good ideas, you're able to share these ideas in a way that's much…”

“the other way to think about sovereignty that we're hearing a lot is that companies want a robust plan for AI. And so, you know, they want options. They, they, they may be using one model, but they actually want to have another model, uh, to be able to, to compare, to benchmark if one model Access gets cut off or too…”

“The thing about general intelligence is you have to solve many different problems to have, you know, the ability to claim general intelligence, and, and fortunately, there are a lot of use cases across Meta, across our family of products, and so that's giving us wonderful material with which to work.”

“In some ways, that's still the hardest question in AI. Um, how do we have different components working together in a very coordinated way?”

“that pivot towards so much energy, literally people energy, as well as computation energy, and progress being devoted to generative model is what's completely changing conversations on this.”

“whether you, you pick this path of, of building AI technology to solve these shorter term problems, or you take an approach that, that is more geared towards general AI. Right now, where we stand, the, the types of problems that we have to solve are very similar. This notion of understanding language, of understanding…”

“LAMA on its own doesn't make, um, for, for the best, ah, the best assistant on, on glasses, because it doesn't have enough of an understanding of the physical world of images, ah, and so on.”