The Ledger, every show

“And my guess is that if somebody put in the effort, they could probably make a superhuman bot for Magic the Gathering now.”

“To then, like, go to a game like Sotos of Catan, it just felt, like, too easy. Like, you could just take a team of five people, spend a year on that, and you'd have it cracked.”

“If somebody's saying something a little weird, because the bot does say weird things every once in a while, their first instinct is not going to be like, oh, I'm talking to a bot. Their first instinct is going to be like, oh, this person is like dumb or distra…”

“You can probably easily 10 X that, you know, I wouldn't be surprised if there's a five hundred million dollar model that's trained in the next year or two.”

“If you train that bot from scratch with no human data it's going to, it could learn to negotiate, but it could learn to negotiate in a language that's not English. It could learn to negotiate in some like gibberish robot language. And then when you stick it in…”

“I don't think you can get an AI to output like the next Harry Potter just yet. That might not be that far off. Maybe it's like five years away or something. But I don't think it's happening just yet.”

“Like how many games does it take for an AI, for a human to become a good chess player or a good diplomacy player or a good artist? The answer is orders of magnitude less than it takes for an AI.”

“Like next, next token prediction is going to, is getting you surprisingly far. But I don't think it's gonna get you all the way there to like replacing, you know engineers at big companies.”

“If we were to add this search, this planning algorithm that would come up with a better strategy when it's actually in the hand, how much better could it do? And the answer was it improved the performance by about a 100,000 X. It was the equivalent of scaling …”

“This wasn't just a matter of scaling compute. It really was an algorithmic breakthrough, and this kind of result would have been doable 20 years ago if people knew the approach to dig.”

“You also do see that like the performance is, is it's not just like a discontinuous jump. It's actually like, you can see the slope of improvement over those hundred million tokens. And so you could probably do some kind of evaluation up to a certain budget an…”

“I think it's a nice eval because there is very little open source code for making poker bots. And there's a lot of published essays, there's a lot of published papers on it, but you really have to reason through everything.”

“Like, with GPT-III, you couldn't scale test time compute. Like, if you gave it a budget of ten million dollars and said, okay, well, let's see what GPT-III can do, it really can't do that much, more than what you could do with, like, 10 dollars or one dollar.”

“I was really impressed with the model's ability to optimize the algorithms that I had developed in my PhD. It was honestly, it was shocking to see how inefficient I was in retrospect, and they were able to make it like, you know, 1000 x faster.”

“One issue you could run into is that you could optimize for certain benchmarks with the routing and then show like, oh yeah, we see this big improvement on these benchmarks. But in real world use cases, it actually ends up not being a significant improvement.”

“When you work on these games, you kind of have to understand the game well enough to like be able to debug your bot because If the bot does something that's, like, really radical and, like, that humans typically wouldn't do, you're not sure if that's, like, a …”

“If you try to do the reasoning paradigm on top of GPT-II, I don't think it would have gotten you almost anything.”

“With Tic-Tac-Toe, we see that, like, GPD-Four .5 falls over. You know, it plays decently well. I shouldn't say it falls over. It does reasonably well. You can draw the board. It can make legal moves, but it will make mistakes sometimes, and if you really need …”

“I think it was around, like, November, twenty-twenty-three, or October, twenty-twenty-three, when I think I was convinced that we had, like, very conclusive signs of life, that, like, oh, this was going to be, this is the paradigm, and it's going to be a big d…”

“I think it's a fair statement to say that these models are less data efficient than humans. And I think that that's an unsolved research question and probably one of the most important unsolved research questions.”

“The way the Poker AI's work today, they're just kind of like sticking to their precomputed GTO strategy. And they're not adapting to the other players at the table.”

“If you come up with an idea, you have to get it passed through legislation and it's a very long process. Computer science is much more exciting in that way because you can just build something. You don't really need permission to do it.”

“The idea of AGI was really science fiction. There were some people that were, you know serious about it, but very few, the majority opinion was that AI was, if anything, it was kind of a dead field.”

“We learned that scale really does work. And in that case, it wasn't scaling, you know, training and neural nets, it was scaling search.”

“But surprisingly, we managed to go, like, the full 40 games without being detected as a bot.”

“If you take out the planning that's being done in AlphaGo and just use the raw Policy network, the raw neural network, it's actually substantially below top human performance.”

“And that planning algorithm that's used in AlphaGo, Monte Carlo Tree Search, is very domain specific. I think people don't appreciate just how domain specific it is because it works in chess, it works in Go, and these have been like the classic domains that pe…”

“What's really interesting about diplomacy, aside from just the natural language component, is that it really is the first major game AI breakthrough in a game that involves cooperation.”

“Eventually I'd imagine that these systems become so strong that, like, it kind of goes the way of chess, where, like, the human's just kind of, like, adding a marginal difference at the end.”

“We did another competition that bought one and that bought Cost under a 150 dollars to train if you were to run it on like a cloud computing service.”

“And I should also say like the way professional poker players train now, They all use bots to assist them. It's a lot like chess where you play the game and then you have a bot analyze your play at the afterwards and see like, okay, did you make mistakes? Wher…”

“I kind of describe poker as essentially high dimensional chess. It's chess. It's like chess where you have to reason about like a probability distribution over actions instead of just like discrete actions.”

“When we released Cicero, we announced it in, like, late twenty-twenty-two, I still found the game, like, really fascinating, and so I, like, kept up with it, I, like, continued to play, and that led to me winning the championship in the World Championship in t…”

“For open AI models, like, they go through a mid-training step, and then they go through a post-training step, and then they're released, and they're a lot more useful. Like, frankly, if you interacted with the only pre-trained model, it would be super difficul…”