why aren't all 19 resolved? a statement only gets an assessment when the public
record can support or contradict it. opinions and what-ifs never can, and 0 checkable
ones are still open, waiting for their date. predictions held up or didn't;
assertions are supported or contradicted. on every card:
▮▮▮▮▮ certainty ·
▮▮▮▮▮ debate potential. speakers are clickable
Assertion Not checkable as stated
Brockman: AI models are reaching parameter counts comparable to human synapses
“It's a hundred T synapses, which kind of corresponds to the weights of the neural net. And so there's some sort of equivalence there. And so we're starting to get to the right numbers. Let me just say that.”
Assertion Not checkable as stated
Brockman: Physicists say GPT-5 re-derived research insights taking months of work
“We've seen physicists starting to kick the tires on GPT-V and say that, like, hey, this thing was able to get, this model was able to re-derive an insight that took me many months worth of research to produce.”
Prediction Not checkable as stated
Brockman: Post-AGI humans will survive without work, but compute will differentiate capability
“And so I think that the question of exactly how, you know, if you don't do work, do you survive?
I think the answer will be yes.
You'll have plenty of material, your material needs met.
But I think the question of
Can you do more?
Can you have not just generat…”
Assertion Not checkable as stated
Brockman: GPT-4 handled multi-turn chat without being trained on it
“We actually did a instruction following post-train on it, so it was really just a data set that was, here's a query, here's what the model completion should be, and I remember that we were like, well, what happens if you just follow up with another query? And …”
Assertion Not checkable as stated
Brockman: Wet lab tests of o3 produced mid-tier journal-level work
“We have wet lab scientists who took models like O-three, ask it for some hypotheses of, here's an experimental setup, what should I do? They have five ideas, They tried these five ideas out, four of them don't work, but one of them does. And the kind of feedba…”
Assertion Partly supported
Brockman: Arc Institute trained 40B DNA model on 13T base pairs
“I'd say that maybe the neural net we produced, you know, it's a 40 B neural net trained on, you know, like 13 trillion base pairs or something like that. The results to be felt like GPT one, maybe starting to be GPT two level, right? It's like accessible or, a…”
Assertion Not checkable as stated
Brockman: GPT-5 performs exceptionally well at front-end software engineering tasks
“GP five is very good at front end, it turns out.”
Assertion Not checkable as stated
Brockman: GPT-5 is OpenAI's most personalizable model to date
“And GPT-V itself is extremely good at instruction following. And so it actually is the most personalizable model that we've ever produced. You can have it operate according to whatever you prefer, just by saying it, just by providing that instruction.”
Assertion Not checkable as stated
Brockman: LLMs consistently generalize to untrained preferences
“In order to get them to be able to operate according to different preferences and values, we just need to show that to them during training, and they are able to sort of generalize to different preferences and values that we didn't actually train against, and …”
Assertion Not checkable as stated
Brockman: OpenAI is compute-limited, preventing further price cuts for now
“Right now we are extremely compute limited, and so I think that if we were to cut prices a lot, it wouldn't actually increase the amount that this model's used.”
Prediction Not checkable as stated
Brockman: AI models will soon get very good at CUDA kernels
“Things like Cuda kernels are a good example of a very self-contained problem that actually our models should get very good at very soon, but it's just difficult because it requires a lot of domain expertise, a lot of like real abstract thinking. But again, it'…”
Assertion Supported
Brockman: OpenAI's robotics team pivoted to build GitHub Copilot
“And we've been through times where, for example, robotics was one in 2018, where we had a great result, but we kind of realized that actually, like, that we can move so much faster in a different domain, right? That, that actually, you know, we had this great …”
Prediction Held up
Brockman: Most AI compute will shift from training to inference
“We're going to move from a world where most of the compute is training the model as we've deployed these models more, you know, more of the compute goes to inferencing them and actually using them.”
Assertion Not checkable as stated
Brockman: OpenAI's 80% o3 price cut yielded neutral or positive revenue
“And you can see it with O three, I think we did like an 80% price cut and actually the usage grew such that it was like, I think in the revenue, it either was neutral or positive.”
Prediction Not checkable as stated
Brockman: Future dev architecture will combine local, remote, and multiplayer agents
“And then you have your codex infrastructure that has a local agent and a remote agent, and that is able to seamlessly, you know, interplay between the two and then is able to multiplayer. Like, this is what the future is going to look like, and it's going to b…”
Assertion Supported
Brockman: OpenAI open-source models saw millions of downloads within days
“Now being used by, you know, there's been millions of downloads of that just over the past couple days.”
Assertion Supported
Brockman: OpenAI Dota used pure RL without human demonstrations
“If you rewind to even 2017, we were working on Dota, which was all reinforcement learning, no behavioral cloning from human demonstrations or anything. It was just From a randomly initialized neural net, you'd get these amazingly complicated, very sophisticate…”
Assertion Not checkable as stated
Brockman: OpenAI's core IMO team was only three people
“The core IMO team at OpenAI was actually three people.”
Assertion Contradicted
Brockman: OpenAI's Dota AI used only 300 million parameters
“And by the way, Dota was like a three hundred million parameter neural net. Tiny, tiny little insect brain, right?”