The Exchanges

Q My question is, so you've built these models with billions of weights. So how does, how do you include the weights in the models? Talk a little bit about that training process. So if the model is going to understand the relationship between these different entities, what do you do to teach it that?

A Um, so, so what we do is we initialize random weights, and then we, like, start to do something called, uh, gradient descent, which is you predict a word and you compute the error between the word you predicted. The model predicted some, like, random word. You tell it you wanted to predict another word, and then you do, and then you try to teach, you update the weights, you update the numbers inside of the weights to basically converge to higher and higher precision. And so this process usually is what we call, it's, it's what happens during pre-training, but we update those weights, um, in order to minimize the distance between what your predicted value is and what the actual value is. Um, and that process usually does some level of convergence and we train it over thousands of GPUs and trillions of tokens. So if you look at something like, um, the eight billion and, and, uh, seventy billion, They were trained on almost 15 trillion, uh, tokens and tokens roughly you can imagine as a word. So roughly like, um, 15 trillion words, which is an incredible outcome. And it requires thousands of GPUs to train it on. And a GPU is, is, you know, um, uh, roughly the cost of like an Audi is what I call it an Audi A three or something like that, but they're very, very expensive. And so like being able to operate these large scale infrastructure, um, Uh, training jobs is quite a feat of bot…

AI assessment note: “we initialize random weights, and then we, like, start to do something called, uh, gradient descent”

Q And just for those unfamiliar, talk a little bit about these weights, you know, a lot of weights involved. What are these weights?

A Um, well, at their simplest level, they're just matrix multiplies, but, um, uh, but, but effectively what you're, what these weights are doing is they are, um, they are encoding or representing human knowledge. So, so in the, in the training process, you are, um, uh, showing the model, uh, lots and lots of, uh, Text information. And, and, uh, what you're trying to do is teach these weights to predict, um, the next word in a sequence of words, in a sentence. So if I say, hello, how are you to, or how are you, and I leave the today out, um, we're teaching the model to build relationships, to learn, uh, relationships between the words to predict the word today. Um, and so when you go through that process, the model starts to, like, learn relationships Between different concepts to learn relationships between different, um, uh, domains of information. And you scale it across all domains. So you're not just learning how are you today, but you're also learning, you know, um, global facts and humanities and mathematics. And you sort of like learn all of this knowledge, uh, and these weights are basically learning to, uh, compute the relationship between these different concepts. And during the alignment process, that's where you can start to teach the model to answer prompts or to behave in a certain way that's more conversational.

AI assessment note: “effectively what you're, what these weights are doing is they are, um, they are encoding”

Q titles, and even a discussion of buying Simon & Schuster, the publishing house, uh, to feed these models. Where are you, do you, do you think in terms of your ability to train these models with, with more data? Because the sense is that the more data they have, the better they're going to be, but They might be hitting a wall in terms of the available data to use.

A You know, I don't think the field really has narrowed down and understands exactly, um, the relationship between, uh, scale and required novelness in, in data. You know, there are techniques that, um, the, the research, the research community is looking at to sort of do better data augmentation, um, synthetic data generation. Um, and so I think it's really, Early to sort of like predict where we will be and what the data situation would look like for improving and enhancing the models. You know, one of the things that we did with Lama three is in post-training, um, we actually leveraged synthetic data. So you'll see, for example, our coding abilities on Lama three is exceptionally high. We're, we're, we're setting kind of a benchmark for what, um, a model can do at the scale that we're at. And Porsche, part of that was like really being innovative and pushing on our ability to do Well, to leverage models to, to improve, um, uh, to generate synthetic data and have synthetic data techniques and approaches to improving the model. So I suspect, um, we'll have some innovations, uh, as we move forward on data, but I don't think we know yet that, you know, we'll run out of data any, or, or that there's some like limiting factor here.

AI assessment note: “I don't think we know yet that, you know, we'll run out of data”

Q You know, as, as you go into the run up of this release, you build a foundational model and what, I mean, you build a model and now you're building it into products. What kind of lift is it to sort of finish that model and then get it operational within products effectively same day of release? How do you do that?

A Um, well, you start, you start very, very early and, and you set the goal for, for the team to do that, to be able to close the loop. But there is a, it's a complex orchestration that's required to sort of move it from model complete to behind a product. You have to, um, you know, we, we have to work across our, our organization is called gen AI, but we, we build the models, but we also deploy them in product. And one of the things that we do, we have to partner very closely with the, um, different app, app teams, application teams like WhatsApp and Instagram and Facebook. And we have very, very close partnership with them and leadership is incredibly involved, um, in moving very quickly. So we're very fortunate to be able to, to have the distribution and the scale and the, uh, And the ability to create these experiences for billions of users to, because we have such a close partnership with these application teams, um, but, you know, rolling a model from, from completion to, to an API requires, or to, to the apps requires that we do a tremendous amount of red teaming and quality checks, and, you know, we have tool use, for example, um, which requires, you know, a system, not just a model, to be designed and rolled behind it. And so we've gotten very, very good at this over the last year, as we move from Lama to, to connect and launching meta AI, our characters, announcing crea…

AI assessment note: “it's a complex orchestration that's required to sort of move it from model complete”

Q hundred billion parameter models, like the one that you're developing and I think is scheduled for release this summer, do you ever like think like we don't want this to be able to be used by everyone because there are inevitably going to be bad actors that use it and We don't necessarily want to make it something that they can use. Like, where is your stance on open source?

A Uh, again, I, I think, uh, while the 400 is still training and we generally, our approach to open source is to like look at the model, apply all of the, the safety critical, um, checks and understand the balances of like the model itself and its performance. So we approach these things very responsibly. Um, but again, the, the, it's too early for me to comment on the, on the 400, uh, plus, um, model or the, the large one. Um, but I do think we, we, it's important to like always remember the benefits of open sourcing. Um, you know, there's both the, uh, you know, the AI advances. I would say like every AI lab in the world today kind of has depended on openness and transparency in order to achieve the outcomes and the results and the improvements to, um, to these models that we have today. And so, um, I think it's always important to like re-anchor on the, on the value and the benefit of open sourcing and, um, these eight and 70 are going to be incredibly useful for people to innovate, um, across the industry and to be able to really push, um, understanding on the science, to be able to understand how to align these models, how to train them, how to improve them, um, which I think is, is really, really valuable.

AI assessment note: “it's too early for me to comment on the 400, uh, plus, um, model”

Q build some personality and, um, and Lama two was a little bit too careful, right? I think that was something that meta has sort of assessed internally and you wanted to make Lama three a little bit more willing to answer questions and have less, I think it's called false rejections. So how do you train a model to be a little bit more of a cowboy on that front?

A Um, I mean, ideally not a cowboy. I think, you know, one of the most important product experience questions that we have, you know, everybody really focuses on, like, general, general knowledge and general capabilities of the systems, and, like, can it, can sort of answer all the questions, but, you know, one of the things that we're excited about at Meta and, and we, we want to be able to innovate on is, is like, there's this idea that you're building, um, alignment for everybody, which is like a system that can, Globally aligned to all humans, but I actually think one of the unique things about our vision is we're, we're like really interested in building AIs for different people for, for different uses. Um, that's why we introduced, uh, our assistant, um, and we believe in personalization for it. That's why we introduced our, um, chatbots, uh, and, and which I believe are kind of like more interest-based and, and aligns to people's interests. Um, and I think core part of that is really How we deal with false reviews is we, we build into the alignment process, the ability to, um, to be steerable, to allow people to sort of, uh, align to and personalize to, to them, and that's how we see our product evolution. I think for the models, the base models themselves, you know, we've done a lot of innovation around, you know, boundary sampling and just making sure that we are, um, wo…

AI assessment note: “we've done a lot of innovation around, you know, boundary sampling”