Assertion Not checkable as stated
Bourgeau: AI progress from pre-training improvements is not slowing down
“It's still remarkable how much progress we're able to achieve in this way, and it's not really slowing down.”
Assertion Not checkable as stated
Bourgeau: Google and DeepMind are actively researching post-Transformer architectures
“I believe so. There's groups doing research on the model architecture side, for sure, within Google and within DeepMind”
Insight
Bourgeau: Architecture and data innovation currently matter more than scale
“The other parts are architecture and data innovation. These also play a really, really important part in the Performance of pre-training and probably even more so than pure scale these days, but scaling is still an important factor as well.”
Assertion Not checkable as stated
Bourgeau: AI development is not running out of training data
“The other part of your question are we running out of data? I don't think so, so there's more.”
Insight
Bourgeau: AI research is shifting to a data-limited paradigm
“I think what might be happening instead is kind of a shift in paradigm where before we were kind of scaling in the data unlimited regime where, where data would scale as much as you would like. And we're kind of shifting more to a data limited regime, which ac…”
Prediction Not checkable as stated
Bourgeau: End-to-end differentiable retrieval and search in training will take years
“I think deep down, I do believe that the long-term answer is to learn this differentiable end-to-end way, which means probably doing pre-training or whatever that looks like in the future, Learn to retrieve as part of the training and learn how to do search as…”
Insight
Bourgeau: AI models must be trained on harmful data to avoid it
“So at a fundamental level, you did, you do need the model to know about those things. So you have to train a bit at least on those so that it knows what those things are and knows to stay away from those, right?”
Prediction Not checkable as stated
Bourgeau: Retrieval-augmented pre-training could become viable in a few years
“I just think it's not unreasonable to think in the next few years, something like that might actually become viable for a leading model like general.”
Insight
Bourgeau: Gemini 3 progress came from small contributions, not one breakthrough
“In my experience, there's maybe one or two of those things that make a larger difference than other things, but it's really a combination of many, many changes and many, many things from a very large team that actually makes Gemini three so much better than th…”
Insight
Bourgeau: Frontier AI development is about building systems, not just neural networks
“We're not really building a model anymore. I think we're really building a system at this point. People have sometimes this view that we're just training a neural network architecture and that's it. But it's really the entire system around the network as well …”
Disclosure
DeepMind's Bourgeau: AI progress is ahead of where I expected
“I think, if I'm being honest with myself, I think we're ahead of where I thought we could go.”
Assertion Not checkable as stated
Bourgeau: Gemini 3's architecture hasn't changed much from Gemini 2.5
“At the high level, I don't think the architecture has changed that much compared to the previous one. It's more of what I was saying before, where a few different things come together to gather, give a large, large improvement.”
Assertion Not checkable as stated
Bourgeau: Gemini answers computer science benchmark questions taking humans significant time
“They are becoming increasingly difficult, and even for me, who has a background in computer science, some of the questions the model answers, it would take me a significant amount of time to answer.”
What-if
Bourgeau: Would not have bet heavily on scaling laws materializing
“I, I'm not sure if I would have bet a lot on, on that actually materializing and being where we are today.”
Prediction Not checkable as stated
DeepMind's Bourgeau predicts major AI-driven scientific breakthroughs within years
“I think we will be able to make some large scientific discoveries in the next few years.”
Prediction Not checkable as stated
Bourgeau: Agentic workflows will accelerate AI research tasks in the next year
“The first part, I think, especially in the next year with more agentic workflows being enabled more and more, that should be able to really accelerate our work there.”
Insight
Bourgeau: AI scale has blurred the line between research and engineering
“I think over time that boundary has blurred quite a lot because we're working on these very large systems now. Research really looks like engineering and vice versa.”
Disclosure
Bourgeau: DeepMind shifted focus from pure research to research engineering
“And I think that's a mindset that has really evolved over the last few years at DeepMind, especially where maybe there was a bit more of the traditional research mindset before, and now with Gemini, it's really more about research engineering.”
Disclosure
Bourgeau works with 150 to 200 people on Gemini pre-training
“So it's a fairly large team at this point. It's a bit hard to quantify exactly, but maybe a 152 hundred people I work on a day-to-day on the pre-training side between data, model, infrastructure, evals, and so coordinating the work of all of these people into …”
Assertion Not checkable as stated
Bourgeau: Early DeepMind research defaulted to synthetic data over real-world data
“And at the time we had to add this from real world data to the name of the project, because people would assume otherwise it would be synthetic environments or synthetic data. And that definitely has shifted completely since then.”
Insight
Bourgeau: Trading peak model performance for lower complexity enables faster progress
“Oftentimes we don't necessarily want to use the best performance version of a research idea, but we'd rather trade off some of the performance for a slightly lower complexity version because we think that will allow us to do more and more progress in the futur…”
Insight
Bourgeau: In deep learning, negative results often mean unoptimized techniques
“Especially in deep learning, a negative results doesn't mean something doesn't work. It means you haven't made it work yet often.”
Disclosure
Bourgeau: Google leadership's research background shields DeepMind from benchmark pressure
“There's actually very little of that. I think because all of the leadership has a research background that they're very much aware that yes, to some extent you can force and accelerate specific benchmarks and certain goals, but in the end, the progress and the…”
Insight
Bourgeau: Pre-training scaling lessons apply directly to RL scaling
“On the RL and RL scaling side, I think we're seeing a lot of the same things we're seeing in pre-training or we saw in pre-training. What's interesting here is because we have the experience of pre-training, a lot of the lessons apply, and we can reapply some …”
