Everything Meta's Generative AI Head said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Ahmad Al-Dahle says Meta AI will be the best free assistant
“And meta AI is going to be the, you know one of the best if not the best assistant that's available for free.”
Meta releases 8-billion and 70-billion parameter Llama 3 models
“We are releasing an updated eight billion parameter model plus a seventy billion parameter model. And these are state of the art.”
Meta leveraged synthetic data during Llama 3 post-training phase
“One of the things that we did with Lama three is in post-training we actually leveraged synthetic data.”
Ahmad Al-Dahle doubts the AI industry will hit a data wall soon
“I don't think we know yet that, you know, we'll run out of data any, or that there's some like limiting factor here.”
Ahmad Al-Dahle: AI models degrade user experience by over-moralizing refusals
“Some of these models, for example tend to do a lot of moralization or like really take a perspective or a point of view. And we worked on and I'm continuing to work on and innovate on how, how the model responds and how it refuses, which I think is also part o…”
Meta's new real-time image generation model produces outputs under one second
“And that's why we did all this model work to really get this thing very fast. It's under a second, which is like really, really exciting.”
Ahmad Al-Dahle says every AI lab depends on open research
“I would say like every AI lab in the world today kind of has depended on openness and transparency in order to achieve the outcomes and the results and the improvements to these models that we have today.”
Ahmad Al-Dahle: 70B parameters is the magical number for billion-user scale
“The seventy billion is a magical number that allows us to scale to billions of people. And so we're really excited that we've been able to strike the right balance between intelligence and efficiency.”
Meta trained Llama 3 8B and 70B models on 15 trillion tokens
“So if you look at something like the eight billion and seventy billion,
They were trained on almost 15 trillion tokens and tokens roughly you can imagine as a word.
So roughly like 15 trillion words, which is an incredible outcome.”
Ahmad Al-Dahle: Meta is currently training a 400-billion parameter Llama 3 model
“We also are talking a little bit about one of the larger models that we're training that is already achieving, you know exceptional performance which is a, it's a model that's over four hundred billion parameters.”
Meta used 100x more compute to train Llama 3 than Llama 2
“So actually, I think it's I believe it's a hundred times more compute.”
Ahmad Al-Dahle admits Meta over-leveraged alignment tools in Llama 2
“In Lama two, it was we definitely I think over leveraged some of the alignment tools to discourage answering those kinds of questions.”
Ahmad Al-Dahle: Llama 3's performance exactly matched Meta's scaling law predictions
“I don't think anything about the model has really personally surprised me in terms of its performance. I think we kind of expected to be here. You know, we do a lot of like rigorous scaling laws and rigorous prediction of what we think the metrics will look li…”
Ahmad Al-Dahle: 8-billion parameter models can run locally on phones
“So an eight billion parameter model is small enough to sort of run, On phones or even laptops at the higher end of those.”
Meta is integrating Meta AI into search suggestions and inboxes
“We have high level entry points directly in the inbox. We're integrating it into search. So we'll have also suggestions and type of heads as you sort of apply it in search. So we're, we've integrated it at a very prominent, in a very prominent way.”
Meta releases safety model Llama Guard and open-sources cybersecurity evaluations
“So if you look at for example, in our release with Lama three, we were open, we're opening a model called Lama guard. And we've also been opening up cybersecurity evals to help understand this, the safety metrics for cybersecurity.”