The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 65 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 2 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Hotz: Meta attracts researchers who want to publish while OpenAI keeps ideologues
“OpenAI can keep ideologues who, you know, believe ideological stuff, and Facebook can keep every researcher who's like, dude, I just want to build AI and publish it.”
O'Laughlin: Google and Meta will see free cash flow drop to zero
“Google, I would argue, is going to free cash with zero. I think Meta will go to free cash with zero.”
Zhang: Meta Failed at Training MoE Models for Llama Series
“The reason why Lama open-sourced the MOE model, because I think they tried to train our MOE model, but they failed. So that, that's why they didn't open source MOE mode for Lama series.”
Howard: Meta 'blew it' on Code Llama due to catastrophic forgetting
“So Code Llama was a, I think it was like a five hundred billion token fine-tuning of Llama II using code. And also prose about code that Meta did. And honestly, they kind of blew it. Because Code Llama is good at coding, but it's bad at everything else.”
Chen: Meta poaching has calmed down and OpenAI came out on top
“I think that met us calmed down a little bit. I think we came out on top”
Morcos: Frontier AI lab data teams are systematically under-resourced
“I think you, what you see in all the frontier labs is that they have data teams. And if you talk to the folks that work on those data teams, what you'll kind of systematically hear is that typically they're under resourced relative to the gains that they're de…”
Morcos: Yann LeCun was never defining Meta's AI strategy
“I don't think he was ever you know, or at least not since the beginning in a role where he was defining AI strategy for Meta. I don't think that's the role he wanted at any point. You know, I think he really wanted to be doing that research, and I think, so I …”
Lambert: Top AI talent is dramatically cheaper than GPU clusters
“Talent is cheaper than GPUs by a dramatic margin, and At the end of the day, it's like, okay, if we're spending this much, they go to the room and they stare in the mirror and you're like, wait, it might not actually be that ridiculous to spend this money on t…”
Lambert: Tulu 3 matches or beats Meta Llama 3.1 on core evals
“On, like, core evals for our Suite of models from, I think, eight, seven D and four or five B is based on llama at the time. It's like it matches or beats meta on these core valves.”
Meta Llama 3.3 and Llama 4 perform poorly on agent benchmarks
“Another, of course, the other surprise was that all the Lama models were not performing well on our benchmark. 3.3 and even the Lama four all were really performing extremely poor.”
Yi Tay: Llama 3 shows Meta may have caught up to Google
“So I think I don't really follow, like, fine much, but I think that, like, Lama Tree actually shows that, like, kind of, like, Meta got a pretty, like, a good stack around training these models you know, like, oh, and I've even started to feel like, oh, they a…”
Future Llama models will match frontier GPT models if progress slows
“As AI progress slows down, so if we get like Llama-IV, Llama-V for example, maybe it's a comparable at that point, like GPT-V or GPT-VI, like, It made it to the point where it was like, look, I just want to use Lama. Like, it's, you know, safe for me to, you k…”
Gomez-Bombarelli: Meta's virtual materials datasets fail to solve real-world chemistry
“Meta has produced tens of millions, hundreds of millions of training data points, but they're all virtual simulations that just don't carry enough water for the thing we actually want to do.”
DeepMind, Microsoft, and Meta are building or using physical science labs
“You see people like Google DeepMind, Microsoft, other places like Meta, either building their own lab or running experiments at someone else's lab to get that data back.”
Lample: Post-training breakthroughs like DPO were impossible without open LLaMA
“And if you look at many of the techniques that were developed after, for instance, Temma was open source, like all these post-training approaches like even DPOD, like performance optimization, all of this were done by people that had access to this model, and …”
Patel: Meta is taking on $40B in debt for its Louisiana AI cluster
“Meta, they've, they're already taking debt on for their largest AI cluster in Louisiana. You know, they're taking like forty billion dollars of debt on for that”
Zhang: Fine-Tuned Llama 3.2 Achieved Superhuman Vision Verification Performance
“We kind of fine-tune our, kind of, for example, NAMA's 3.2 with our, kind of, verification, human annotated verification data. We get, kind of, superhuman performance on these two verification tasks, and then we do not need human on these two tasks. Let's furt…”
Swyx: Priscilla Chan will have more impact on humanity than Zuckerberg
“My quick hot take in, in a single sentence is if Priscilla Chan gets Half of what she wants to do done. She will have more impact on humanity than Mark Zuckerberg, right? The Facebook will just be a funding mechanism for the greatest bio research work done in …”
Morcos: Meta's metaverse bet will pay off in the long run
“I think the one that's still really up in the air is a metaverse, but I would actually argue that I think that's going to end up paying off in the long run. I think the Ray-Ban glasses pretty darn cool. And a lot of the foundations of what was in reality labs …”
Morcos: Zuckerberg personally approved high-risk AI training datasets at Meta
“When I was at Meta, certainly legal stuff around data sets was very challenging and becoming increasingly challenging, and there are a number of situations where, you know, the only person that could approve things was Zuck because of the scale of the risk, I …”
Palazzolo: Meta should focus on app integration over frontier models
“Maybe what's actually good for their P&L and their finances is to not focus so much on building these, like, insane, huge, state-of-the-art models, which they've obviously struggled with more recently, but it's to take things that are, you know, 70, 80, like, …”
Lambert: Meta withholding its leading benchmark model is bad execution
“But to be a model that claims to be open and then not release the model that is your leading claim is just, like, that is, like, bad execution.”
Shah: Meta and LinkedIn keep social and professional graphs aggressively closed
“Right now our information, all of us nodes are in the social graph at Meta or the professional graph at LinkedIn, both of which are actually relatively closed and actually very annoying ways. Like very, very closed, right? Especially LinkedIn.”
Shawn Wang doubts Meta's Llama Stack will win broad developer adoption
“I've been a little bit more doubtful on Lama stack. I think you've been more positive. Basically, it's just like the meta version of whatever HuggingFace offers, you know, or TensorRT, or BLM, or whatever the open source opportunity is. But like, to me, it's n…”
Howard: Meta's schedule-free optimizers are not that exciting
“I mean, I don't care very much, honestly. Like, I don't think that schedule-free optimizer's that exciting. It's fine.”
Meta used stepwise reward models and Monte Carlo Tree Search for Llama 3.1
“They actually went the extra step to, no pun intended, to actually train stepwise reward models. That's kind of crazy, no? I mean, they wanted each step in the chain of thought to be so good that they actually took the extra effort to train step, to train step…”
Meta changed Llama 3's pre-training data mixture mid-training run
“What happened is we changed the data mix during the training of Lama three with some findings that happened in the... Training is long, so you have to do something while it's training. And what the team did, I was working on my side of motion post-training, bu…”
Scialom: Meta Used Llama 2 to Filter and Tag Llama 3 Pre-Training Data
“LAMA was the best, at the time, before LAMA Free, the best model we had access to legally, to labelize the web and select what are the good tokens and the bad tokens. The additional thing is that it also enabled to have a topic tag, Like, is it about law? Is i…”
Yi Tay: Meta's Llama is corporate open weights, not grassroots open source
“To me, Lama Tree is like... Meta has an org that is hypothetically very similar to Gemini or something but they just decide to release the weights It's open weights It's open weights and everything”
Tay: Google and OpenAI built general models three years before academia
“Places like Google and Meta, OpenAI, we will be working on things, like, Three years ahead of everybody else, and then suddenly, like, then Academia would be, like, still working on, like, these task-specific things.”
Chintala: Meta built FAIR because AI capabilities rate-limited Meta's product development
“And for, then, like, the thesis was very simple. It was, like, AI is currently rate-limiting Meta's ability to do things. Our ability to build various product integrations, moderation, various other factors. Like, AI was the limiting factor, and we just wanted…”
Firshman: Llama 2 costs $25M to train but $50 to fine-tune
“Lama II as a base model is that, like, yeah, it costs twenty five million dollars to train to start with, but then you can fine tune it for, like, 50 bucks.”
Only Tech Giants Will Build In-House AI Infrastructure Long-Term
“I think a lot of these companies over in the long run, like, you know, they're accepted maybe super big ones, like, you know, the Facebook and Google, they're always gonna build their own ones, but like everyone else, like some extent, you know, I think they'r…”
Patel: Broadcom is working on Meta's second-generation internal AI chip
“So Broadcom is, is, is, is working with, on Google's chip right now, but of course on Meta's, Meta's internal AI chip, which they're on the second generation of, working on that.”
Park: Facebook data better captures human baseline state than LinkedIn or Twitter
“If we were to look at purely social media, like if you really, you know, if I were, you know, if I had to really pick, Facebook likely is interesting because I actually do think it is most sort of a default version of people because you go to LinkedIn, it's ve…”
Kulik: Academic Labs Must Avoid Problems Solvable by Brute-Force Compute
“For sure, Microsoft, Meta, those ones are kind of like the companies that have basically infinite resources, and as an academic, I don't have infinite resources, you know, but we have an interest in problems that, you know, haven't crossed the radar of those c…”
Ravi: Meta's SA-Co Benchmark Has Over 200,000 Unique Concepts
“If you look at the size of these benchmarks, the previous benchmark, Peng Chuan mentioned, Elvis, that everyone uses, it has about 1.2 K unique concepts and the benchmark that we created, which we're calling segment anything with concepts or Seiko, COCO for sh…”
Ravi: Over 70% of SAM 3 Dataset Annotations Are Negative Phrases
“We have about 70, more than 70% of the annotations are these like negative phrases that are not present in the image.”
McCloy: Meta AI is a 'silent monster' driven by distribution reach
“Meta AI, for example, which I don't think gets necessarily remarked upon a ton in the like AI enthusiast, AI developer community, but like it's a silent monster, right? Because of Meta's distribution and reach.”
Meta research: Sub-1B language models benefit more from depth than width
“For example, they find that depth is more important than width, so it's more important to have models that have, like, more layers than just making them more wide. They also find that GQA helps, that tie-in the embedding helps, so I think it's a nice study ove…”
Robinson: Original DiT Research Showed Compute Scaling Outweighed Hyperparameters
“This is so interesting because the original diffusion transformer paper from Facebook actually showed that, in fact, the specific hyperparameters of the transformer didn't really matter that much. What mattered was that you were just increasing the amount of c…”
PyTorch was originally built for researchers without considering production requirements
“PyTorch actually started as the framework for researchers. Don't care about production at all.”
Cheah: AI community will replicate Meta's pipeline scheduling algorithm
“This weird scheduling, which I'm quite sure people are going to start replicating it, is to reduce the bubble, the wastage.”
Scialom: Llama 3 Scaled Pre-Training to 15 Trillion Tokens
“It's the same recipe done in terms of architectures and training than LAMA-II, but we put so much effort on scaling the data and the quality of data. There's now 15 triant tokens compared to two triants, so it's another magnitude there as well, including for t…”
Scialom: Meta expanded Llama 3's vocabulary size to support multilingual capabilities
“Lama III compared to Lama II is multilingual, has multilingual capabilities. We worked on that. And so, because you have languages that are not just Latin languages like English, there's a lot of different characters you want to include them to represent, like…”
Meta skipped coding, reasoning, and multilingual annotations for Llama 2
“And we didn't annotate at all for code, neither for reasoning or multinguity.”