The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 25 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Brockman: Scaling neural nets will unlock qualitatively new behaviors
“And I think that what that's going to unlock is you're going to be able to scale up these models and you're going to see qualitatively different behaviors from what you've seen so far.”
Brockman: Solid Engineers with Zero AI Experience Are Productive Day One at OpenAI
“So someone like that can actually become productive from day one.”
Brockman: Keep Machine Learning Cores Minimal and Surround Them with Engineering
“You actually try to make that core be as small as possible because machine learning is really hard. It's a lot of compute. It's really hard to tell what's going on there. And so you want it to be as simple as possible, but then you surround it by as much engin…”
Brockman: Lacking Linux support and APIs is the biggest barrier to AI progress
“The thing about League of Legends is it doesn't run on Linux and it doesn't have a game API and little things like that actually are the biggest barrier to making AI progress in a lot of ways.”
Brockman: AI research will be automated before other human jobs
“I think it's actually not AI researcher. The AI researcher will kind of go before. It's actually very interesting when you ask people this question. I think that everyone tends to say whatever their job is as well as the hardest one. But I actually think that …”
Brockman: AI hardware will advance faster than people expect
“Now, if you look forward to what's going to happen over upcoming years is the hardware for these applications for running neural nets really, really quickly are going to get fast, faster than people expect.”
Brockman: ML projects require tracking experiment inputs over outcome milestones
“Initially the way that the project management was happening was that each week, well, so we'd written down our milestones of let's beat this person by this state, let's beat this other person by this state, let's, you know, be able to do, you know, kind of the…”
Brockman: Machine learning dramatically increases the leverage of human programmers
“And like the way that, that I now think about machine learning systems is that they're really a way to make the leverage of human programmers go way up.”
Brockman: AI can discover non-obvious strategies that transfer to humans
“The bot had, like, taught him the strategy that he could use against a human and I think that was, like, very interesting and a good example of the kinds of things that you can get out of these systems, that they can discover these very, sort of, you know, Non…”
Brockman: OpenAI character-prediction model learned state-of-the-art sentiment classification
“At OpenAI, we see this sometimes, for example, we had a paper on this unsupervised learning where you train a language model You train a model to predict the next character in Amazon reviews, and just by learning to predict the next character in Amazon reviews…”
Brockman: Specialized brain-like hardware architectures will run AI models insanely faster
“Now, if you move to specialized hardware that is sort of much more brain-like and that runs a bunch of you know, that sort of runs in parallel with a bunch of tiny little cores that, that you're going to be able to run these models sort of insanely faster.”
Brockman: OpenAI chose Dota 2 after League lacked Linux support and API
“Actually the way that we selected the game was we went on, on Twitch and just looked down the list of most popular games in the world and starting, you know, number one is League of Legends. The thing about League of Legends is it doesn't run on Linux and it d…”
Brockman: ML researchers copy files over Git because experiments must run side-by-side
“Because the thing is, if you have a new idea for, okay, well, I've kind of got this thing working, and now I'm going to try something slightly different. As you're doing the new thing, well, machine learning is, to some extent, very binary. At the start, it ju…”
Brockman: Behavioral cloning algorithms imitate actions rather than true intent
“It's very clear, like when you're just doing cloning that like these algorithms like learn to imitate what it sees rather than the actual intent.”
Brockman: Never trust absolute probability numbers in AI, only their trend
“The probability with these things, you never really trust the probabilities. You just trust the trend of the probabilities.”
Brockman: ML designers should reserve model capacity for unscriptable problems
“At the end of the day that I think a lot of our job as the system designers here is to push as much of that model capacity and as much of the learning Towards the like interesting parts of the problem that you can't script, that you can't possibly do any proce…”
Brockman: Machine learning cores must be understood behaviorally, not just logged
“Normally, the way that you do it is, well, your goal is to have everything be very observable. And so, yeah, you want to put metrics on everything, and like, you know, if something's not understandable, add more logging, like, you know, that that's how you des…”
Brockman: Game-based AI research will precede real-world problem-solving deployments
“And I think what you're going to see is that there's a lot of work that's going to be done in games, but the goal is, of course, bring it out of the game, and actually use it to solve problems in the real world, and to actually, you know, be able to interact w…”
Brockman: Emergent sentiment feature disappears in slightly smaller models
“And that this effect goes away if you use a slightly smaller model.”
OpenAI downloaded 2TB of Dota 2 replays daily before filtering
“Originally, we were downloading all of them every day, and realized that was about two terabytes worth of data a day. That adds up quite quickly, so we ended up filtering down to the sort of most expert players.”
Brockman: OpenAI put all compute into one final run before tournament
“Two weeks before the International was kind of our cutoff for at this point, there's not much more we can do, that we're gonna do our biggest experiment ever, put all of our compute into one basket, and see, see where it goes.”
Brockman: OpenAI Dota bot lost to Pajkatt due to an unseen wand build
“Well, I, I'd say very, very specifically that kind of the root cause here was that he had gone for an item, an early wand build. And we had just never done item early wand build. And so it's just like our bot had just never seen this particular item build befo…”
Brockman: OpenAI stitched two bots together hours before facing Dota pros
“Monday bot, it is pretty good at the first wave. This new bot is a super bot thereafter. ... So we wrote, so, so we already, we had some code for doing something similar. So we kind of revived that. And then in the three hours, Jay spent his time doing a very …”