The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 37 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Mensch: AI export controls will not stop European or Asian progress
“Effectively, if there's some export control over weight this is not going to stop any country in Europe, any country in Asia to continue its progress. And they will collaborate to actually accelerate that progress.”
Mensch: Mixtral matches Llama 2 70B performance at one-sixth the cost
“Mixtral is actually on par with Lama-to-seven TB while being approximately six times cheaper or six times faster for the same price.”
Mensch: AI regulation should target applications, not foundational math
“What you want to regulate is the application, and the issue we had, and the issue we're still having now is, We hear a lot of people saying we should regulate the tech, so we should regulate the function, the mathematics behind it, but really you never use a l…”
Mensch: FLOP count is the wrong metric for regulating AI models
“Pre-market conditions like flops, the number of flops that you do to create a model is definitely not the right way of doing of measuring the performance of a model.”
Mensch: AI will impact every country's GDP by double digits
“It will have an impact on GDP of every country in the double digits in the coming years.”
Mensch: General-purpose base AI models will eventually become open source
“There are general purpose models like base models, compression of the web. That are eventually going to be open source and that can serve as the right basis for constructing specialized systems.”
Mensch: Nations and enterprises must own their AI cultural alignment
“For me, this is an inherent limitation of centralized AI models, where you're thinking that you can encode some universal values and some universal expertise into a general purpose model. At some point you need to take the general purpose model and ask a speci…”
Mensch: Mistral Sabah outperforms Arabic AI models five times its size
“Today our model it's the 24 B. It's called Mistral Sabah. It's a model tune in Arabic. Is outperforming every other language model that are, like, five times larger.”
Mensch: Closed-source AI models are unfit for high-certainty applications
“You can evaluate a model much better if you have access to the weights than if you only have access to APIs. And so if you want to build certainty around The fact that your system is going to be a hundred percent accurate, I don't think you should be using a c…”
Mensch: 2020-2021 AI research suffered from flawed scaling laws in GPT-3 and Gopher
“There was also a misconception on GPT-free and basically in 20, 21, every paper made this mistake.”
Mensch: Compute-optimal LLM scaling requires equal relative growth in parameters and data
“In common words, if you multiply by four your compute capacity, you should multiply by two, the model size and by two, the data size.”
Mensch: Pre-trained AI models should be neutral without creator bias
“Pre-trained models should be neutral, and we should empower our customers to take these models and just put their editorial approaches, their instruction, their constitution, if you want to talk like entropy, Into the model. So that's the way we approach the t…”
Mensch: Internal Mistral models rank among top three globally
“Internally We have stronger models that are in between 3.5 and four that are basically second or third, the second or third best model in the world.”
Mensch: Open-source AI trails proprietary models by six months
“So really we think that the gap is closing. The gap is approximately six months at that point.”
Mensch: Open-source models will equal proprietary AI performance
“But I really think that will converge to a setting where you have proprietary models and the open source models are just as good.”
Mensch: Fine-tuning access makes GPT-4 easy to exploit into bad behavior
“It's actually super easy to exploit an API. It's super easy, especially if you have fine tuning access to make GPT-IV behave in a very bad way.”
Mensch: AI companies must run on distinct product and scientific frequencies
“You have fast frequencies on the product side. It's iterating every week. And you have slow frequencies on the science side that are looking at why profoundly the product is failing on certain domains and how they could fix it through research, through new dat…”
Mensch: Asynchronous AI workloads will drive massive new compute demand
“We are moving to, towards workloads that are more and more asynchronous. So workloads where you give a task to an AI system, and then you wait for it to do, like, 20 minutes of research before returning. So that's definitely changing a bit the way you should b…”
Mensch: Overtraining models past Chinchilla limits lowers inference costs
“If you take into account the fact that your model Should also be efficient at inference time. You probably want to go far beyond the Cinchilla scaling low. So it means you want to overtrain the model. So train on more tokens than would be optimal for performan…”
Mensch: Mixture of experts decouples model capacity from inference cost
“A sparse mixture of experts, you take the dense layer and you duplicate it several times. And so that's where you actually increase the number of parameters. So you increase the capacity of the model without increasing the cost. So that's the way of decoupling…”
Mistral AI plans to monetize through an open-core business model
“As a business, we do need to have a valid monetization approach at some point. But we've seen many businesses build open core approaches and have, A very strong open source community, and also a very good offer of services, and that's what we want to build.”
Mensch: Fine-tuned Mistral 7B matches GPT performance on specific enterprise tasks
“They took Mistral-Seven-B, had a lot of human annotations, had a lot of proprietary data, just modify Mistral-Seven-B so that it solved their task, just as well as GPT-PT-PT-PT, but only for a lower cost and a higher level of control.”
Mensch: Hugging Face DPO outperformed Mistral AI's initial instruct model
“The hugging face folks first did the direct preference optimization on top of Mistral seven B and made a very strong, a much stronger model than the instructive model we proposed at the early release.”
Mensch: Modern AI models are essentially compressions of internet data
“The current generation of modern models that we are using today are not that much are not much more than just a compression of whatever is available on the internet.”
Mensch: LLMs function as programming languages, not standalone products
“If you look at what the LLM does, it's not really different from a programming language. It's actually used very much as a programming language by the application makers. There's a strong confusion made between what we call a model and what we call an applicat…”
Mensch: AI is a general-purpose technology redefining software construction
“It is a general purpose technology. Because it basically revisits entirely the way we are building software and the way we are using machines.”
Mensch: Small nations and firms should buy horizontal AI, build vertical
“If you're a small enterprise or a small country, you should probably buy. And what is vertical and specific to you? And that's definitely something that you need to build.”
Mensch: Mixtral has 46B total parameters but executes 12B per token
“You have eight experts per layer and you execute only two of them. So what it means at the end of the day is that you have a lot of parameters on your model. You have forty six billion parameters, but the thing is that the number of parameters that you execute…”
Mensch: AI industry stopped publishing open research after GPT-3
“And all of a sudden in with GPT-free, this tide kind of reversed and companies started to be more opaque about what they were doing because they realized there was actually a very big market. And all of a sudden in 20, 22, on the important aspects of AI and on…”
Mensch: Mistral must build frontier models to stay relevant to developers
“Really at the end of the day developers look at performance and latency. And so that's why we think that as a company, we need to be very much on the frontier if we want to be relevant.”
Mensch: LLM training is 100,000x less data efficient than the human brain
“If you compare like the training process of a large language model to the brain, you have like a factor I think 100,000.”
Mensch: We will not understand machine reasoning anytime soon
“We are not going to know about how machines reason anytime soon.”
Mensch: Everyone will use specialized AI models within five years
“What we think is that fast forward five years everybody will be using their specialized models within parts of complex applications and systems.”
Mensch: Mistral AI helps France's unemployment agency match job seekers
“We're working, for instance, with the French unemployment system to actually connect opportunities of jobs to unemployed, unemployed people through AI agents that are being actionated by, obviously, human operators within the agency.”