Lambert: OLMo 3 7B outperforms Meta's Llama 3.1 8B in internal tests
Nathan Lambert · Open Source AI Strikes Back — Inside Ai2’s OLMo 3 ‘Thinking" · Nov 20, 2025 · at 4:54
Nathan Lambert is an AI research scientist at Ai2. He highlights internal benchmark results comparing Ai2's open OLMo 3 7B model to Meta's widely used Llama 3.1 8B.
0:00 / 0:08exact quote · 9.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And I just think of this cause like Lama 3.1 AP is one of the most used models and hugging base of all time. And this should be better. We're, In our measurements, we see it as being better than Llama.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Nathan Lambert
Lambert: Chinese open AI models currently do not contain backdoors
“Like, you can't prove that the models aren't doing certain backdoors, where I'm fairly certain they definitely aren't now.”
Lambert: AI progress will yield steady improvements rather than rapid singularity
“I think these researchers are going to grind out improvements for multiple years, but never in a way that results in this kind of accelerating well that we get drawn into.”
Lambert: OLMo 3 32B base model matches Qwen 2.5 32B quality
“This base model is similar in quality to the best available, which is like Quinn's 2.5, 32 B is, was still the best base model.”
Lambert: 80% of a16z's open-model portfolio startups use Alibaba's Qwen
“80% of companies building with open models are using Quinn, which is like 16 to 24% of his portfolio, which is still a lot.”