AlphaGo

“AlphaGo running on GPUs had an ELO score of about 3200. Lease it all was about 35, 35 50. And I might be getting the first digit wrong. It might be 2000 instead of 3000. But like, you know, it was more than 200. And then when put on on TPUs, it actually went t…”

“In the second game, there was this famous move called move 37, which was creative. It was original. It actually wasn't completely original. It was a one in 10,000 game move. It had been in the canon of games that we're trained on. But when we went back and pla…”

“And I think of the prominent AI leaders, I think Demis does a pretty good job of actually Translating the technology into more quantifiable and tangible ways for, like, your every person, because I think there's a lot of focus on, like, benchmarks and a lot of…”

“I believe, for example, in areas like mathematics, because it's a closed system you don't need data. You can just make conjectures and see if you can prove them and keep on like that. In that sense, it's a bit like AlphaGo where you can play against yourself.”

“Really you can think of a lot of The things we're doing today, all the leading models with thinking modes and chain of thought reasoning as aspects of what was sort of pioneered with AlphaGo coming back now.”

“And I think a lot of those ideas, both from AlphaGo and AlphaZero are really, really relevant to where we are with today's foundation models. And I think a lot of that is what we're going to see of the advances the next few years.”

“The lesson I've learned from all the alpha projects we've done, specifically alpha go and alpha fold is I think the techniques we have and the problems I look like to look for are great in, if this, if the situation can be described as massive combinatorial se…”

“That is going to happen in every single domain. It will happen in science, in math, in physics, in chemistry, it's going to happen in material science, it's going to happen in biology, it's going to happen in healthcare, drug discovery, but it may also even ha…”

“You have to break the combinatorial wall, and the way to do it is to add deep learning guidance. It's actually very similar to the principles that analyze something like AlphaGo or AlphaZero.”

“It's interesting to see like Codex's personality shine through when writing code. I would say it does the most stuff that humans don't do kind of in this alpha go sense where it's like, oh, it'll write a Python script to like modify some part of the file syste…”

“The best South Go player in the world is a South Korean, and he was about 15, I think, when AlphaGo match happened. He's in his mid-twenties now, and he's by far the strongest player there's ever, Been by the ELO ratings, because he's learned natively, young e…”

“And in move 37, the computer does something, and right away the announcers are like, oh my. Oh, what is that? And Lisa Dole, you can literally, they show him, like, sweating and thinking, and he's like, what the hell just happened? They go, and they go, I thin…”

“He's actually a pretty young looking guy, but the crazy thing is, as he starts to put the pressure on the Chinese guy, They cut the feed in China. No way, really? How badass is that? They cut the feed of the broadcast. They're like, no, we will not show, we wi…”

“And I think if we had done it a few months earlier, we would have probably lost.”

“It's a five-game thing. It just won the first three games straight. I mean, completely cleaned up. And with inventive, new, creative moves that no human has played before.”

“To some extent super intelligence in that sense has already been achieved. So right, AlphaGo was a super intelligent system, and there were other systems during that time that were built that were super intelligent in narrow domains.”

“My point is that, like, this is where the AlphaGo analogy breaks down. And, not necessarily breaks down, but, like, it's not going to be as easy as self-play was in AlphaGo.”

“We had versions of AlphaGo and AlphaZero with the thinking turned off, so it was just the model telling you its first idea. And, you know, it's not bad, it's maybe like master level, something like that, but then if you turn the thinking on, it's way beyond wo…”

“What they did with AlphaGo and AlphaZero, as you mentioned it showed, as I recall, something you would take 5000 times as much training to match what you were able to do with still a lot of training and the inference time compute that you were doing with Go.”

“There was almost nothing that went superhuman except for something like AlphaGo.”

“Just understanding the world's information, and then trying to sort of almost compress that into your memory, that's not enough for solving a novel math problem, or a novel, novel conjecture. So there, you know, we start needing to bring in, I think we talked …”

“Historically, if you look at AlphaGo, which actually followed quite closely the recipe of you pre-train your model on all human data, you then use RL to make it better, and then you do some search at inference time, the compute there was very skewed for the mi…”

“I think it was watched live by a hundred million people.”

“AlphaGo had the ability to think 50 to 60 moves ahead, helping the model to win game two.”

“And, you know, in many ways, all of the subsequent breakthroughs, whether they were learning to play Atari games or AlphaGo or AlphaFold or many of the health applications that we did that produced best in the world class of x-ray recognition or ophthalmology,…”

“What we discovered, however, was that once we then made AlphaGo more generally available, the humans actually used AlphaGo to improve their performance.”

“If you take out the planning that's being done in AlphaGo and just use the raw Policy network, the raw neural network, it's actually substantially below top human performance.”

“And with AlphaGo, the scoring, because of the scoring inherent to Go, AlphaGo could generate lots of new moves and instantly see which ones are good. So it could iterate its way through that learning process to get to really, really good moves. And so then you…”

“Problems that are where you can tell the computer the rules, and you can give it a score so it knows whether it's doing well, which is what you got with AlphaGo. It would be very difficult to do that writing a novel, because there's no set of rules, and there'…”

“AlphaGo, I think it was roughly 45% of the research cited on their AlphaGo paper came from the University of Alberta.”

“If you look at the four or five matches of AlphaGo, there were multiple moments when AlphaGo made a movement. Master Li Sado was really surprised. And if you look at the Go community, people were just amazed by the kind of Creativity AlphaGo has in terms of ma…”

“So we've already seen that with our, with AlphaGo in the match with Lee Sedol. So as you probably know, there was a famous move in the second match against Lee Sedol, move 37, where all the commentators, all these sort of nine Dan masters were saying, saying, …”

“And so what DeepMind did with AlphaGo was to actually combine some of the classic AI, you know, game search with deep learning, with this type of you know, neural network approach to do the evaluation.”

“So AlphaGo, the first thing that AlphaGo did was it learned from all the existing, the entire existing database of Go matches played by human masters. That was the first thing it did was learn from those, right? Thirty million moves or something like that is t…”