Llama 3

“So if you look at something like the eight billion and seventy billion, They were trained on almost 15 trillion tokens and tokens roughly you can imagine as a word. So roughly like 15 trillion words, which is an incredible outcome.”

“So actually, I think it's I believe it's a hundred times more compute.”

“Some of these models, for example tend to do a lot of moralization or like really take a perspective or a point of view. And we worked on and I'm continuing to work on and innovate on how, how the model responds and how it refuses, which I think is also part o…”

“I don't think anything about the model has really personally surprised me in terms of its performance. I think we kind of expected to be here. You know, we do a lot of like rigorous scaling laws and rigorous prediction of what we think the metrics will look li…”

“We are releasing an updated eight billion parameter model plus a seventy billion parameter model. And these are state of the art.”

“We also are talking a little bit about one of the larger models that we're training that is already achieving, you know exceptional performance which is a, it's a model that's over four hundred billion parameters.”

“Like to train Lama three meta use synthetic data, like data created from basically, you know, from AI itself”

“When we were pulling Llama three together, we had really pulled in all the research, all the every, we pulled out every single stop we had and unwittingly kind of killed the pipeline. So researchers, you know, the way that it works is you build a base And you'…”