Pineau: Compute and Data Scale AI Linearly, While Algorithms Cause Step-Functions
Joelle Pineau · Cohere's Chief AI Officer, Joelle Pineau: Why Scaling Laws Will Continue & Future of Synthetic Data · 20VC with Harry Stebbings · Nov 3, 2025 · at 10:07
Joelle Pineau, Chief AI Officer at Cohere and former VP of AI Research at Meta, explains how AI capabilities advance across hardware, data, and algorithmic dimensions.
0:00 / 0:29exact quote · 29.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Compute and data have a more linear effect on progress. You build more compute. You run bigger models. You can typically get better performance. You feed in more data. It's not just quantity. You need to worry about quality and diversity as well, but roughly it's more linearish with respect to the data. The algorithms are the ones that have the nonlinear effect, and so you can explore lots of ideas, and then something like the transformer comes along and just changes the paradigm.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Joelle Pineau
Pineau dismisses catastrophic AI existential risk predictions as unscientific
“I don't have a lot of patience as a scientist for people who are predicting the extremist scenarios, the catastrophic risks of AI, you know, AI becomes our overlord kind of scenario.”
Pineau: Out-of-the-box reinforcement learning will not deliver AGI
“Now, you know, where we're maybe getting a little bit ahead is thinking that just RL out of the box is gonna give us AGI. That part, a lot less so. You know, if you look at the curve of progress, RL is terribly inefficient, and so the amount of signal you need…”
Pineau: Reinforcement learning is fundamental to AI and will not disappear
“Oh, I'm still super bullish on RL in that, like, the concept itself is so fundamental. You know, this idea of training through a system of rewards, of indicating what's valuable and what's not valuable through numerical values, like, that is so fundamental. It…”
Pineau: Using reinforcement learning to teach AI social behavior remains unsolved
“RL, to shape the behavior of models, to get them to be social creatures, that we have no idea how to do. I mean, I don't know if you have children, but like shaping their behaviors, you know, the number of times you can repeat the same thing, and still they do…”
Pineau warns against betting that AI compute scaling will hit a wall
“They've been, the scaling laws have been remarkably robust. They don't play exactly as we expect, but still, they've been remarkably robust. Lots of people have bet against scaling laws in the past. And I would say overall, you know, we've seen a pretty, prett…”