Lean

“So what it really enables is that you can now write neural networks essentially in Lean. So instead of writing in, like, PyTorch, it's like a PyTorch-like abstraction, but you can, like, kind of, you know, write it in Lean, and so it can be fully formalized in…”

“So lean still has a lot of shortcomings there. It's CPU based and you know, it's not, Like, getting that onto the GPU has a lot of nuances there. So, you know, a lot of work needs to be done. So what we've started with is a framework, you know, making that mor…”

“We're talking about, for example, the grind tactic in Lean. It can currently handle a lot of mass proofs, like, at a very low level. And this is pretty shocking because I have seen, you know, actually another company working in the same space, like, you know, …”

“I think a Lean-based system will struggle in those very creative places, which is why we at Axiom actually also invest on something called mathematical discovery.”

“If you want proof to be informal math, It's very annoying, because then that's, like, just makes objective function. Your code is something like Python, your proof is, say, natural language, math proof. You will not have very strong RL kind of performance, rig…”

“Currently, actually, you know, for each line of code written, there could be like 20 lines of proof.”

“Auto-informalization is a lot easier than auto-formalization minus the problem of no grounding, right?”

“So we just released AXLE, A-X-L-E, stands for Axiom Lean Engine. And it's really a set of, kind of, proof validation and manipulation tools that are built for Lean in the language of Lean. So it's a bunch of metaprogramming tools.”

“And Excel is currently, I think, 14 like, such tools starting from Verify Proof, which is the sort of, To make sure that there's not, nothing weird you know, going on, like no, no sort of cheating by lean code.”

“And we have seen also, we have heard from a lot of the people that Claude plus Axel is kind of their go-to setup for now.”

“I think auto-generated Blueprint is going to be a technical bottleneck that many people are trying to solve around the same time.”

“Now that we're in this regime where you can just get ChatGPT to tackle thousands of questions at the same time, and it will return proofs for a significant fraction of them. Now, actually the onus is back on the humans to verify all the outputs. And so, yeah, …”

“Because it is a, you know, lean based system, it is really good at sort of routine bookkeeping, and it will actually choose a lot of the more mechanistic, you know, arguments over the ones that require like a clever, say one picture solution.”

“There's also auto-informalization, which is kind of translate back, I mean, from Lean to English. That's easier than auto-formalization, because most of the machines' AI have seen a lot more English than Lean.”

“I think the sort of gap between, say, for example, Lean and another, like, strongly typed language like Rust is a lot closer than the gap between Lean and English.”

“It's only two-digit million number of tokens out there in the open, open world.”

“We are actually gonna release them on a public, like, API on these, all these dozen of pools. Very, very soon. Beginning of March.”

“Almost all of the new school of mathematicians are accepting Lean.”

“Using this program called Lean that is a formal math proof verifier, and there's rumors that Google's building their system to leverage that a lot. OpenAI apparently went way, way down just the text-based LLM reasoning path and had a lot of success there.”

“What surprised me is this time they don't use formal language, but instead they just use LM. And so last year when they tried to do the IMO, they need like a like a human to kind of translate the natural language. Problems to Lean, and then they use Lean to ki…”

“And so if you can just use lean to kind of become the verifier, then it's easy. So I think we probably will see very powerful reasoning models in, in math very soon.”

“For Lean there are not enough data for Lean, right? There are, like, probably one million tokens in about Lean. Right now, and I know a lot of, like, professors and PhD researchers are trying to build the biggest lean data set in the future.”

“It's I think definitely possible. Yeah. Like he, so the professor is Kevin buzzard and he got like a grant and now he just like focused on writing the proof for Ling. Like he's hoping to finish that in like two or three years. And then basically like if Ling i…”