OpenAI researcher Mehtaab Sawhney explains to a16z partner Lisha Li how OpenAI's reasoning initiative, Astra, resolved an open problem regarding high-dimensional sphere packing.
Assertion Not checkable as stated
Sawhney: OpenAI models prune search trees instead of brute-forcing proofs
“You can sort of look at it, and it's reasoning like a mathematician, and because it knows a few very correct bits, it makes the right decisions and is eventually able to prune the search tree. It's not really trying everything. It tries a lot of different thin…”
Opinion
Sawhney: AI models update proof path likelihoods better than humans do
“The model somehow is much better able to, like, it seems, for several of the solutions we've seen, somehow it seems much better able to update the solution, like, how likely the path is to work, like, versus rejecting a path versus a human doing it.”
Assertion Supported
Sawhney: Astra's proof of the sphere packing bound is a few pages
“I think also in general, it was one of these solutions which, I knew several people had tried the problem, it's pretty remarkable because, like, the model solution, especially for this being, like, the LP can't do better than this, was, like, quite short. It's…”
Insight
Sawhney: Solving harder math problems by definition demonstrates better AI taste
“I tend to be pretty utilitarian in my view of taste, and, like, if you're able to solve problems faster by making better judgments, like, I think that's, like, the best, like, general proxy I have for taste, and somehow the fact that solving harder problems me…”
Prediction Not checkable as stated
Sawhney: AI will produce exponentially more math, making it easier to absorb
“Along, I mean, of course models are going to help us produce exponentially more mathematics, but they also make it much easier to absorb it and right now, okay, it's still a bit of a challenge back and forth, but I think it's, for me at least much, much faster…”
Insight
Sawhney: Backtracking is a general-purpose reasoning tool, not math-specific
“A lot of these behaviors that we're describing mathematically, like backtracking, or kind of starting again, I mean, these are not really specific to mathematics. I mean, we're seeing them specifically in mathematics in these examples, but kind of, they're gen…”