DeepMind

“Turns out through my experience, the best research scientists, also very good engineers, like very good engineers. And we've noticed like through DeepMind and OpenAI is just like the companies that made the most progress over like last six years or five years …”

“Demis was way more in the kind of robotics land and sci-fi land. I mean, he was fully thinking that, you know, the way to manage the economy, the way to make economic decisions was to simulate the entire economy.”

“That's part of the reason why we started the Alpha Fold project, because it was actually my group that was looking around for other applications of DQN-like, AlphaGo-like tools, and in a hackathon that we did one week someone stumbled across, across this probl…”

“And it was his work in, I think he, I think for his PhD thesis, he put together like 80 definitions of intelligence. And aggregated those into a single formulation, which was how do we you know, the intelligence is the ability to perform well across a wide ran…”

“If I were to sort of summarize the core thesis of deep mind, it was that it would be possible to the motivation for generality was that you would be able to learn you know, a rewarding behavior in one environment and transfer in a more compressed or efficient …”

“Over the last decade, the amount of compute that we've used to train the largest models in the world has increased by an order of magnitude every single year. So I went back and had a look at the Atari DQN paper that we published in. And that used just two pet…”

“Peter Thiel, you know, to his credit did actually have significant vision here, although he sold pretty early, I think, and now doesn't seem to be in the game.”

“Basically the story was that everybody was training models on too few tokens because of the paper from 20 20 that happened to be not very well executed.”

“One was that the Gemini project was formed as a result of having two sort of Parallel efforts on LLMs, mostly led by Google Brain and what we now call Legacy DeepMind. So earlier in the year, there was an effort to merge the two projects, and that's when sort …”

“I think that one of the things that was so remarkable to me about AlphaFold II is, initially it was a really small team, you know, three, and then later, 18 people or so, and they solved what was kind of a fifty-year grand challenge in biology, which is a pret…”

“All they need to do is do a zero day on Slack. And then they can know what DeepMind is up to in very high fidelity and OpenAI and XAI and others.”

“Dota V and AlphaStar were near super intelligent systems, and if OpenAI and DeepMind had sunk more compute into them, they would have definitely become super intelligent.”

“So I would bet on XAI. I think they're just very hungry and mission-oriented in a way that gives them a lot of really unique advantages.”