Nov 8, 2017 · 1h 0m · y-combinator
Building Dota Bots That Beat Pros - OpenAI's Greg Brockman, Szymon Sidor, and Sam Altman · Y Combinator
Greg Brockman · 34m spoken Szymon Sidor · 16m spoken Sam Altman · 2m spoken Craig Cannon · 2m spoken
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this Y Combinator interview, OpenAI leaders Greg Brockman, Szymon Sidor, and Sam Altman discuss computational hardware scaling, software engineering infrastructure, and machine learning breakthroughs behind OpenAI's Dota 2 bot. They share candid insights into research methodologies, high-stakes tournament matches against elite professionals, and the broader path toward artificial general intelligence.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The partners as informed peer 1.1 Guest teaching 1.4 Guest disagreement 0.1 The partners pushing back 0.1
speaking balance: gold is the partners, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 49:27 Szymon corrects Sam Altman on human player adaptation
When Sam suggests humans only beat the bot by discovering exploits, Szymon directly reframes the premise, stating that players actually learned to match the bot's exact mechanical skill level.
Hardest push from the partners ▶ 24:11 Sam Altman challenges the team's shifting calibrationSam playfully calls out the inconsistency of the team's confidence, pointing out that their nightly messages swung wildly between guaranteed defeat and complete victory.
Biggest teaching moment ▶ 31:32 Szymon explains model modification to Craig CannonSzymon corrects the misconception that black-box models are entirely uncontrollable, explaining clearly how sampling distributions are adjusted to force exposure to novel strategies.
The partners hold their own ▶ 5:45 Sam Altman synthesizes hiring misconceptionsSam actively reinforces and sharpens the discussion by citing frequent applicant misunderstandings regarding AI PhD requirements versus solid foundational software engineering.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The partners as informed peer | Guest teaching | Guest disagreement | The partners pushing back | Why |
|---|---|---|---|---|---|---|
| Scaling Neural Nets and the Future of AI Hardware | 1 | 2 | 0 | 0 | Sam Altman and Craig Cannon ask broad opening questions regarding scaling neural nets and underexplored AI domains. Greg Brockman and Szymon Sidor provide comprehensive, educational answers about hardware scaling and batch sizes. | |
| Transitioning Beyond von Neumann Hardware Architectures | 1 | 2 | 0 | 0 | The hosts ask foundational questions about hardware innovation and how non-PhD engineers can break into AI. Greg and Szymon explain the difference between von Neumann architecture and brain-like parallel compute. | |
| Origins and Infrastructure Setup of the Dota 2 Bot Project | 1 | 1 | 0 | 0 | Sam inquires about the division between machine learning science and software engineering on the Dota project. Greg and Szymon explain that the bulk of the effort was pure systems engineering and infrastructure setup. | |
| Building Python Environments and Machine Learning Workflows | 0 | 2 | 0 | 0 | Craig asks why Python and TensorFlow were chosen. Greg delivers a deep-dive monologue on transpiling Lua, setting up gRPC protocols, and the idiosyncratic workflows of ML experimentation. | |
| Reinforcement Learning and Self-Play Dynamics | 1 | 2 | 0 | 0 | Craig asks how bot feedback loops work in reinforcement learning. Szymon demystifies RL and self-play, recounting how a week of self-play beat months of scripted bot development. | |
| Fine-Tuning Creep Blocking and Scoreboard Milestones | 1 | 0 | 0 | 0 | Craig asks when OpenAI decided to compete at The International. Greg explains how TrueSkill scoreboard tracking and changing milestone management enabled consistent progress. | |
| Benchmarking Against Pros Leading Up to The International | 2 | 1 | 1 | 1 | Sam presses the guests on how capable the bot actually was right before the tournament, noting how wild their win-probability estimates swung nightly in text updates. | |
| KeyArena Matches, The Magic Wand Bug, and Pro Reactions | 1 | 2 | 0 | 0 | Greg shares anecdotes from the KeyArena locker room and their match loss to Pajkatt due to an unseen Magic Wand item build. The dynamic is storytelling with attentive host prompts. | |
| Human Exploits, Swarm Testing, and Preparing for 5v5 | 1 | 2 | 0 | 0 | Craig asks how engineers manipulate a black-box neural network after discovering exploits. Szymon and Greg explain sampling probabilities and feature engineering such as teleport visibility. | |
| The Franken-Bot Emergency Fix Before Facing Arteezy | 1 | 1 | 0 | 0 | Greg and Szymon explain the emergency creation of the Franken-bot when their updated model exhibited accidental baiting behavior hours before facing Arteezy. | |
| Defeating Arteezy and Sumail, and Teaching Humans New Strategies | 1 | 1 | 0 | 1 | Sam jokes about whether the engineers got full nights of sleep, prompting Szymon to detail their grueling 6am all-nighters and Azure quota negotiations before beating Sumail. | |
| Post-Tournament Live Streams and Human Adaptation | 1 | 2 | 1 | 0 | Sam asks if humans beat the bot by finding exploits. Szymon corrects him, explaining that dedicated pro players actually match the bot's mechanical precision over hundreds of games. | |
| Core Engineering Skills and Engineering Culture at OpenAI | 2 | 1 | 0 | 0 | Szymon and Greg outline core engineering requirements at OpenAI, emphasizing short, bug-free code over academic research pedigrees. Sam humorously interrogates who writes the fewest bugs. | |
| Non-Technical AI Roles, Sustainable Productivity, and AI Hype | 2 | 1 | 0 | 0 | Craig asks how non-technical people can participate in AI, while Sam conducts a lightning round covering overwork, startup AI hype, and the hardest human jobs to automate. | |
| Why Video Games Matter for AGI and OpenAI Recruitment | 1 | 1 | 0 | 0 | Craig asks about the connection between video games and AGI. Greg outlines why virtual game simulations provide safe, complex baselines for advancing general intelligence. |
Statements from this episode (36)
Brockman: AI hardware will advance faster than people expect
“Now, if you look forward to what's going to happen over upcoming years is the hardware for these applications for running neural nets really, really quickly are going to get fast, faster than people expect.”
Brockman: Scaling neural nets will unlock qualitatively new behaviors
“And I think that what that's going to unlock is you're going to be able to scale up these models and you're going to see qualitatively different behaviors from what you've seen so far.”
Brockman: OpenAI character-prediction model learned state-of-the-art sentiment classification
“At OpenAI, we see this sometimes, for example, we had a paper on this unsupervised learning where you train a language model You train a model to predict the next character in Amazon reviews, and just by learning to predict the next character in Amazon reviews…”
Brockman: Emergent sentiment feature disappears in slightly smaller models
“And that this effect goes away if you use a slightly smaller model.”
Sidor: AI research neglects understanding existing methods and limits
“And what people do is people kind of try to invent problems and the, such as solving some complicated games of character structure, and they try to Add kind of extra features to their models to combat those problems, but I think there's very little research ha…”
Sidor: Rigorous baseline optimization will advance AI more than complex architectures
“And you know, it's not the kind of sexy research that people want to see, where you have like some hierarchy of big RNN, but it actually, this kind of research, I think at this point will advance field the most.”
Brockman: Specialized brain-like hardware architectures will run AI models insanely faster
“Now, if you move to specialized hardware that is sort of much more brain-like and that runs a bunch of you know, that sort of runs in parallel with a bunch of tiny little cores that, that you're going to be able to run these models sort of insanely faster.”
Sidor: Strong Engineering Is Much More Valuable at OpenAI than Writing Exotic Models
“So, so essentially becoming a good engineer for our team is much more valuable than, for example, people spending you know, months upon months implementing exotic models in TensorFlow.”
Brockman: Solid Engineers with Zero AI Experience Are Productive Day One at OpenAI
“So someone like that can actually become productive from day one.”
Brockman: Keep Machine Learning Cores Minimal and Surround Them with Engineering
“You actually try to make that core be as small as possible because machine learning is really hard. It's a lot of compute. It's really hard to tell what's going on there. And so you want it to be as simple as possible, but then you surround it by as much engin…”
Brockman: OpenAI chose Dota 2 after League lacked Linux support and API
“Actually the way that we selected the game was we went on, on Twitch and just looked down the list of most popular games in the world and starting, you know, number one is League of Legends. The thing about League of Legends is it doesn't run on Linux and it d…”
Brockman: Lacking Linux support and APIs is the biggest barrier to AI progress
“The thing about League of Legends is it doesn't run on Linux and it doesn't have a game API and little things like that actually are the biggest barrier to making AI progress in a lot of ways.”
OpenAI downloaded 2TB of Dota 2 replays daily before filtering
“Originally, we were downloading all of them every day, and realized that was about two terabytes worth of data a day. That adds up quite quickly, so we ended up filtering down to the sort of most expert players.”
Brockman: ML researchers copy files over Git because experiments must run side-by-side
“Because the thing is, if you have a new idea for, okay, well, I've kind of got this thing working, and now I'm going to try something slightly different. As you're doing the new thing, well, machine learning is, to some extent, very binary. At the start, it ju…”
Sidor: OpenAI's RL bot beat its hand-coded bot after one to two weeks
“So I leave, there is nothing, I come back, there is this reinforcement learning bot, and actually, it's beating our scripted bot after, like a week worth of engineering effort. Possibly it was two weeks, but it was something very miniature compared to the deve…”
Brockman: Behavioral cloning algorithms imitate actions rather than true intent
“It's very clear, like when you're just doing cloning that like these algorithms like learn to imitate what it sees rather than the actual intent.”
Brockman: ML projects require tracking experiment inputs over outcome milestones
“Initially the way that the project management was happening was that each week, well, so we'd written down our milestones of let's beat this person by this state, let's beat this other person by this state, let's, you know, be able to do, you know, kind of the…”
Brockman: OpenAI put all compute into one final run before tournament
“Two weeks before the International was kind of our cutoff for at this point, there's not much more we can do, that we're gonna do our biggest experiment ever, put all of our compute into one basket, and see, see where it goes.”
Brockman: Never trust absolute probability numbers in AI, only their trend
“The probability with these things, you never really trust the probabilities. You just trust the trend of the probabilities.”
Brockman: OpenAI Dota bot lost to Pajkatt due to an unseen wand build
“Well, I, I'd say very, very specifically that kind of the root cause here was that he had gone for an item, an early wand build. And we had just never done item early wand build. And so it's just like our bot had just never seen this particular item build befo…”
Sidor: OpenAI ran 50-computer LAN party where humans found bot exploits
“There was a point after the event where we set up this big LAN party where we had, like, 50 computers running the bot. We kind of unleashed this swarm of humans to kind of add our bot. And they find, found all the exploits”
Brockman: Machine learning dramatically increases the leverage of human programmers
“And like the way that, that I now think about machine learning systems is that they're really a way to make the leverage of human programmers go way up.”
Brockman: ML designers should reserve model capacity for unscriptable problems
“At the end of the day that I think a lot of our job as the system designers here is to push as much of that model capacity and as much of the learning Towards the like interesting parts of the problem that you can't script, that you can't possibly do any proce…”
Sidor: OpenAI's Dota bot required daily overnight updates to beat pros
“Because actually it's a little known fact, but from day to day, like each version of the experiment was not good enough to beat the next player, next, next day's professional.”
OpenAI Dota bot learned baiting behavior without explicit reward incentives
“It was like one of the major examples, one of the major examples of the things that we kind of didn't have explicit incentive for, and yet the bot actually learned them.”
Brockman: OpenAI stitched two bots together hours before facing Dota pros
“Monday bot, it is pretty good at the first wave. This new bot is a super bot thereafter. ... So we wrote, so, so we already, we had some code for doing something similar. So we kind of revived that. And then in the three hours, Jay spent his time doing a very …”
Brockman: OpenAI's hybrid bot defeated pro player Arteezy 3-0
“We first played our TZ who showed up on, on our switch bot, you know, kind of the Franken bot. And you know, that beat him three times.”
Brockman: Machine learning cores must be understood behaviorally, not just logged
“Normally, the way that you do it is, well, your goal is to have everything be very observable. And so, yeah, you want to put metrics on everything, and like, you know, if something's not understandable, add more logging, like, you know, that that's how you des…”
Brockman: AI can discover non-obvious strategies that transfer to humans
“The bot had, like, taught him the strategy that he could use against a human and I think that was, like, very interesting and a good example of the kinds of things that you can get out of these systems, that they can discover these very, sort of, you know, Non…”
Brockman: OpenAI Dota bot went undefeated 5-0 against Sumail
“With Sumail, we went undefeated. I, and I think it was five zero that day.”
Sidor: Pro Dota player reached 30% win rate against OpenAI bot
“I think it might be actually 30. And that player played hundreds of games with the bot.”
Sidor: OpenAI sacrifices code modularity for brevity to reduce bugs
“We sometimes actually kind of sacrifice good engineering habits, good, good kind of code modularity to make our code shorter and simpler and kind of having less, essentially less lines where you can make bugs.”
Sidor: Machine learning math is easier to learn than good engineering
“Getting good basics in linear algebra and in basic statistics, that's, especially when Doing experiments, it's easy to make like elementary statistics mistakes and linear algebra is just kind of most of what you need to know to like basic optimization as well …”
Brockman: AI research will be automated before other human jobs
“I think it's actually not AI researcher. The AI researcher will kind of go before. It's actually very interesting when you ask people this question. I think that everyone tends to say whatever their job is as well as the hardest one. But I actually think that …”
Brockman: Game-based AI research will precede real-world problem-solving deployments
“And I think what you're going to see is that there's a lot of work that's going to be done in games, but the goal is, of course, bring it out of the game, and actually use it to solve problems in the real world, and to actually, you know, be able to interact w…”