The Ledger, every show

“The challenge that we see with some of our competitors is that they're investing billions or hundreds of billions into creating assets that are deprecating very fast because those are commodities.”

“There's no such thing as like one system that is going to be solving all of the problems of the world. And so at the end of the day, or you just don't believe in that context at all. It's never going to exist. I mean, there's, you have a wealth of problems, ju…”

“It's also the only way in which you can create systems that are effectively using your, the folklore knowledge of your employees. That's the knowledge that you've accrued for decades. The only way in which to turn it into an asset that nobody gets access to is…”

“Effectively, if there's some export control over weight this is not going to stop any country in Europe, any country in Asia to continue its progress. And they will collaborate to actually accelerate that progress.”

“It looks like around flipping. Yes. I don't know about that deal particularly, but it makes sense from both perspectives.”

“Assuming that the model should be well behaved is, I think, a wrong assumption. You need to make the assumption that the model should know everything. And then on top of that, have some modules that moderate and guardrail the model.”

“And the way you make this healthy competition is not By trusting a couple of companies to do their own safety, it's rather for, it's rather the way you do it is to ask application makers to comply with some rules.”

“Mixtral is actually on par with Lama-to-seven TB while being approximately six times cheaper or six times faster for the same price.”

“What you want to regulate is the application, and the issue we had, and the issue we're still having now is, We hear a lot of people saying we should regulate the tech, so we should regulate the function, the mathematics behind it, but really you never use a l…”

“Pre-market conditions like flops, the number of flops that you do to create a model is definitely not the right way of doing of measuring the performance of a model.”

“Nothing is showing that LLM is actually marginally better than a search engine to find knowledge on, on topics that would enable bad use.”

“Today going, banning open source, preventing it from happening is really a way, well, to enforce regulatory capture, even though the actors that would benefit from it, Don't want it to happen. But by design, if you actually ban small actors from doing things i…”

“No scientific studies in proper form was published, but then policy papers started to cite Non-scientific papers arguing that these were scientific evidences that the bioweapon narrative was actually true. And then policy papers started to cite the other polic…”

“I don't think AI is going to be the one triggering the next pandemic. It's always going to be climate change.”

“If we can make a model which is growingly intelligent, then maybe you're at a singularity level. There's no evidence whatsoever that we are on the way of doing that, of making that happen.”

“It's mostly an efficient way of training models to have bigger models that are used as teachers for smaller models, but it's not enough. And so you also need human signal.”

“Inherently this is a technology that is going to get commoditized. The reason for that is that it's actually not hard to build. You have around 10 labs in the world. That know how to build that technology, that get access to similar data that follows the same …”

“And so the acceleration of technological progress is I think where most of the value creation will be. It will take a little bit of time. And it will be less measurable less predictable than the efficiency gains that AI is going to produce. But the two things …”

“Well, if you look at the trends in 20, 24 I'd say there might have been like a six month gap. If you look at the trend in 20, 25, I think the gap is more around three months. So I guess it's up to anyone else, anyone to guess what the gap is going to be next y…”

“Basically you have a saturation effect when you pre-train models around 10 to the power of 26 flops. The reason for that is that there's only that much data you can find to compress when you pre-train models.”

“And the next two years is going to be about taking model and making them extremely good at a certain skill set. And that's actually more exciting because we are getting to a point where if you pick a domain can just make it a superhuman, but we are not going t…”

“As if you're an independent country, you want to have independent defense systems. And if you want to have independent defense systems, you will need them to, you will need your own independent artificial intelligence because this is making it into the defense…”

“Like we released the first Sparse mixture of experts back at the beginning of 2024, and they built on top, and they released deep seek free, and deep seek was built on top of that. Well, it was, it's the same architecture, and we released, like, everything tha…”

“Their best way of accessing the U S markets is by just giving the things for free. And so it does make sense. It's a very natural thing to do to build a business in China, which is protected. Then to export the thing for zero.”

“And so what's going to happen, I think in the next 10 years is that all of the manufacturing processes will be rebuilt around LLM orchestrators.”

“We're never going to be able to build systems that work out of the box in a single shot. And the one thing that we try to convey to our customers is that they need to build a prototype that's going to work 80% of the time. But then how do they get from 80% to …”

“And so our bet in robotics and what we've been doing with multiple companies in defense in particular is to build that platform that allows to train models fit to purpose that can then be deployed on the edge potentially”

“Eventually the entire economy is going to run on AI systems. That's for sure. But it might take up 20 years because it's actually fairly complex.”

“It will have an impact on GDP of every country in the double digits in the coming years.”

“There are general purpose models like base models, compression of the web. That are eventually going to be open source and that can serve as the right basis for constructing specialized systems.”

“For me, this is an inherent limitation of centralized AI models, where you're thinking that you can encode some universal values and some universal expertise into a general purpose model. At some point you need to take the general purpose model and ask a speci…”

“Today our model it's the 24 B. It's called Mistral Sabah. It's a model tune in Arabic. Is outperforming every other language model that are, like, five times larger.”

“You can evaluate a model much better if you have access to the weights than if you only have access to APIs. And so if you want to build certainty around The fact that your system is going to be a hundred percent accurate, I don't think you should be using a c…”

“A team of five is faster than a team of 50. Except if you organize the team of 50 to be 10 teams of five that are sufficiently uncoupled.”

“Gemini was a bit too slow, and I think they recovered sufficiently well since.”

“And so I think that's the end state is models are effectively going to be A starting point for any AI application developer. They need to be surrounded by tools, by lifecycle management platform basically, and that's the one thing that we started to build.”

“Like general purpose models are a bit undifferentiated, but the differentiation that you need to create for your application comes from the data you put into it, the user feedback that you gather and the intelligence that you have to figure out what the applic…”

“There's obviously compute, but given the amount of data you have we have at hand compute is already running into is no longer the bottleneck. The bottleneck is more the data at that point. You should look at text to text models.”

“And actually these vertical models are not going to be out there. They're going to be built by the application makers because the only way you can Make a low latency model that is super good at a specific task is to get rid of the general purpose aspect becaus…”

“NVIDIA is, at that point. The cloud providers are pretty much at cost LLM providers, we're not at cost hopefully, but the margin that, Are known to be lower than the typical software margins.”

“I don't think there's any way in which the marginal cost and the margin of the most important part of that technology, which is really the foundational layer becomes zero because otherwise there's definitely going to be I guess fairness problem.”

“So if you look at the way we train models from three years ago, and the way we train model today, I think we have probably made something around a hundred times algorithmic improvement. So that's the, that's probably where most of the gains were actually made …”

“A shortcut to distribution is to create demand through open source models.”

“Capital is equal compute. Then compute is correlated with quality. It's not completely dependent on it.”

“So, I mean, we're growing our compute like every companies. We are convinced that we don't need to grow at the same rate because there's a lot of barriers that are not compute related that are appearing on the way and that we're already seeing”

“European funds were unstructured to do the kind of deal that we were proposing. So we didn't even have a lot of conversation cause they just couldn't get their head around the investment that needed to be made as well as we were a pre-revenue company.”

“We could not operate only from Europe, and that we needed to go to the US very quickly.”

“Is it too late? I wouldn't recommend going into the foundational layer business. I know Sam didn't recommend to do that one year ago, and we did, and it seemed to have so far changed a few things. So I wouldn't be I think it would be arrogant for me to say tha…”