Prediction Not checkable as stated
Ahmad Al-Dahle says Meta AI will be the best free assistant
“And meta AI is going to be the, you know one of the best if not the best assistant that's available for free.”
Disclosure
Meta releases 8-billion and 70-billion parameter Llama 3 models
“We are releasing an updated eight billion parameter model plus a seventy billion parameter model. And these are state of the art.”
Disclosure
Meta leveraged synthetic data during Llama 3 post-training phase
“One of the things that we did with Lama three is in post-training we actually leveraged synthetic data.”
Opinion
Ahmad Al-Dahle doubts the AI industry will hit a data wall soon
“I don't think we know yet that, you know, we'll run out of data any, or that there's some like limiting factor here.”
Insight
Ahmad Al-Dahle: AI models degrade user experience by over-moralizing refusals
“Some of these models, for example tend to do a lot of moralization or like really take a perspective or a point of view. And we worked on and I'm continuing to work on and innovate on how, how the model responds and how it refuses, which I think is also part o…”
Assertion Supported
Meta's new real-time image generation model produces outputs under one second
“And that's why we did all this model work to really get this thing very fast. It's under a second, which is like really, really exciting.”
Assertion Not checkable as stated
Ahmad Al-Dahle says every AI lab depends on open research
“I would say like every AI lab in the world today kind of has depended on openness and transparency in order to achieve the outcomes and the results and the improvements to these models that we have today.”
Insight
Ahmad Al-Dahle: 70B parameters is the magical number for billion-user scale
“The seventy billion is a magical number that allows us to scale to billions of people. And so we're really excited that we've been able to strike the right balance between intelligence and efficiency.”
Assertion Supported
Meta trained Llama 3 8B and 70B models on 15 trillion tokens
“So if you look at something like the eight billion and seventy billion,
They were trained on almost 15 trillion tokens and tokens roughly you can imagine as a word.
So roughly like 15 trillion words, which is an incredible outcome.”
Disclosure
Ahmad Al-Dahle: Meta is currently training a 400-billion parameter Llama 3 model
“We also are talking a little bit about one of the larger models that we're training that is already achieving, you know exceptional performance which is a, it's a model that's over four hundred billion parameters.”
Assertion Partly supported
Meta used 100x more compute to train Llama 3 than Llama 2
“So actually, I think it's I believe it's a hundred times more compute.”
Disclosure
Ahmad Al-Dahle admits Meta over-leveraged alignment tools in Llama 2
“In Lama two, it was we definitely I think over leveraged some of the alignment tools to discourage answering those kinds of questions.”
Assertion Not checkable as stated
Ahmad Al-Dahle: Llama 3's performance exactly matched Meta's scaling law predictions
“I don't think anything about the model has really personally surprised me in terms of its performance. I think we kind of expected to be here. You know, we do a lot of like rigorous scaling laws and rigorous prediction of what we think the metrics will look li…”
Assertion Supported
Ahmad Al-Dahle: 8-billion parameter models can run locally on phones
“So an eight billion parameter model is small enough to sort of run, On phones or even laptops at the higher end of those.”
Disclosure
Meta is integrating Meta AI into search suggestions and inboxes
“We have high level entry points directly in the inbox. We're integrating it into search. So we'll have also suggestions and type of heads as you sort of apply it in search. So we're, we've integrated it at a very prominent, in a very prominent way.”
Assertion Supported
Meta releases safety model Llama Guard and open-sources cybersecurity evaluations
“So if you look at for example, in our release with Lama three, we were open, we're opening a model called Lama guard. And we've also been opening up cybersecurity evals to help understand this, the safety metrics for cybersecurity.”
