The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 14 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Mensch: AI safety should regulate application makers, not trust big model labs
“And the way you make this healthy competition is not By trusting a couple of companies to do their own safety, it's rather for, it's rather the way you do it is to ask application makers to comply with some rules.”
Mensch: AI base models should know everything and use modular guardrails
“Assuming that the model should be well behaved is, I think, a wrong assumption. You need to make the assumption that the model should know everything. And then on top of that, have some modules that moderate and guardrail the model.”
Mensch: Closed-source AI wastes billions in duplicated compute
“We think that it's too early, and we think it's really, ah, damaging for the science, ah, to actually move into such an opaque regime, where you have A few companies basically doing the same thing, just not communicating about it spending billions of compute d…”
Mensch: Viable AI agents require a 100x reduction in compute costs
“I think making a model smaller is definitely a way to make agent work. Because one problem you have with agent that very quickly is start to, if you run an agent on GPT four you're going to run out of money very quickly. And so if you divide by a hundred the, …”
Mensch: Distilling state-of-the-art small models requires training massive models first
“The other thing about moving into larger models is that it enables you to train smaller models that are better, which is through variety of techniques like distillation or synthetic data generation. So this, these two things are quite related. If you want to m…”
Arthur Mensch: Mistral Aims to Make AI Safer Through Openness and Scrutiny
“By doing what we do, by being much more open about the technology we create, we want to steer the community into a regime where things just work better, where things are safer because put under more scrutiny, and really our intention there is to, well, to take…”
Mensch: Mistral 7B proved AI model compression limits are far away
“What we showed with Mistral Seven B is that we weren't, we were definitely far away from the limit of compression.”
Mensch: Base AI models should know everything, with guardrails added externally
“Assuming that the model should be well behaved is, I think, a wrong assumption. You need to make the assumption that the model should know everything. And then on top of that, have some modules that moderate and guardrail the model.”
Gil: Mistral reached near GPT-4 capability within one year of founding
“They went from basically starting the company to almost GPT-IV level in less than a year.”
Mensch: Mistral Is Not Yet the Top Expert in Instruction Tuning
“We're not the top experts in the world in, in, in making good instruction fine tune models. We're definitely ramping up and the team is, is getting better and better at that.”
Mensch: Single H100 GPU can serve hundreds of API customers
“It's going to be less costly because just a single H-Handroid can serve hundreds of customers.”
Mensch: AI agents currently suffer from repetitive mode collapse
“What we see with agent is mode collapse. So not very interesting mode collapse. They start repeating themselves and they fall into loops.”
Mensch: Pure scientific model performance ignores crucial runtime inference costs
“And if you want to push the performance, the pure performance of models, you don't care about inference because you, well, you are not going to use the model. You're just going to see whether they're good or not. And that's really for scientific purposes. But …”