Brockman: ML researchers copy files over Git because experiments must run side-by-side
Greg Brockman · Building Dota Bots That Beat Pros - OpenAI's Greg Brockman, Szymon Sidor, and Sam Altman · Y Combinator · Nov 8, 2017 · at 16:07
OpenAI co-founder Greg Brockman describes why traditional software engineering version control practices differ from empirical machine learning research workflows.
0:00 / 0:34exact quote · 34.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Because the thing is, if you have a new idea for, okay, well, I've kind of got this thing working, and now I'm going to try something slightly different. As you're doing the new thing, well, machine learning is, to some extent, very binary. At the start, it just, like, doesn't work at all, and you don't know why. Or it kind of works, but it has some weird performance, and you're not sure exactly, is it a bug? Is it just how this data set works? Like, you just don't know. And so if you've gotten it working at all, and then you make a change, you're always going to want to go back and kind of compare to the previous thing you had running. And so you actually do want the new thing running side by side with the old thing. And if you're constantly stashing and unstashing and checking out and whatever, then you're just going to be sad.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Greg Brockman
Brockman: Scaling neural nets will unlock qualitatively new behaviors
“And I think that what that's going to unlock is you're going to be able to scale up these models and you're going to see qualitatively different behaviors from what you've seen so far.”
Brockman: Solid Engineers with Zero AI Experience Are Productive Day One at OpenAI
“So someone like that can actually become productive from day one.”
Brockman: Keep Machine Learning Cores Minimal and Surround Them with Engineering
“You actually try to make that core be as small as possible because machine learning is really hard. It's a lot of compute. It's really hard to tell what's going on there. And so you want it to be as simple as possible, but then you surround it by as much engin…”
Brockman: Lacking Linux support and APIs is the biggest barrier to AI progress
“The thing about League of Legends is it doesn't run on Linux and it doesn't have a game API and little things like that actually are the biggest barrier to making AI progress in a lot of ways.”
Brockman: AI research will be automated before other human jobs
“I think it's actually not AI researcher. The AI researcher will kind of go before. It's actually very interesting when you ask people this question. I think that everyone tends to say whatever their job is as well as the hardest one. But I actually think that …”