OpenAI researcher Mehtaab Sawhney contrasts AI model backtracking during mathematical proofs with human mathematicians' cognitive bias of prematurely downgrading difficult approaches.
Assertion Supported
Sawhney: Astra proved the asymptotic linear programming bound for sphere packing
“And what the model shows is that Actually the linear programming bound in large dimensions has this extremely nice asymptotic behavior, and the proof kind of explains where this is coming from, and because you understand this LP bound perfectly, this actually …”
Assertion Not checkable as stated
Sawhney: OpenAI models prune search trees instead of brute-forcing proofs
“You can sort of look at it, and it's reasoning like a mathematician, and because it knows a few very correct bits, it makes the right decisions and is eventually able to prune the search tree. It's not really trying everything. It tries a lot of different thin…”
Assertion Supported
Sawhney: Astra's proof of the sphere packing bound is a few pages
“I think also in general, it was one of these solutions which, I knew several people had tried the problem, it's pretty remarkable because, like, the model solution, especially for this being, like, the LP can't do better than this, was, like, quite short. It's…”
Insight
Sawhney: Solving harder math problems by definition demonstrates better AI taste
“I tend to be pretty utilitarian in my view of taste, and, like, if you're able to solve problems faster by making better judgments, like, I think that's, like, the best, like, general proxy I have for taste, and somehow the fact that solving harder problems me…”
Prediction Not checkable as stated
Sawhney: AI will produce exponentially more math, making it easier to absorb
“Along, I mean, of course models are going to help us produce exponentially more mathematics, but they also make it much easier to absorb it and right now, okay, it's still a bit of a challenge back and forth, but I think it's, for me at least much, much faster…”
Insight
Sawhney: Backtracking is a general-purpose reasoning tool, not math-specific
“A lot of these behaviors that we're describing mathematically, like backtracking, or kind of starting again, I mean, these are not really specific to mathematics. I mean, we're seeing them specifically in mathematics in these examples, but kind of, they're gen…”