Mensch: AI model training achieved 100x algorithmic improvement over three years
Arthur Mensch · Arthur Mensch: Open vs Closed - Who Wins and Mistral's Position | E1146 · 20VC with Harry Stebbings · Apr 29, 2024 · at 22:41
Arthur Mensch, co-founder and CEO of Mistral AI, argues that algorithmic efficiency gains far outpace raw hardware compute cost reductions in AI development.
0:00 / 0:17exact quote · 17.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So if you look at the way we train models from three years ago, and the way we train model today, I think we have probably made something around a hundred times algorithmic improvement. So that's the, that's probably where most of the gains were actually made in the last three years.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Arthur Mensch
Mensch: Cloud investments in AI startups resemble round-tripping
“It looks like around flipping. Yes. I don't know about that deal particularly, but it makes sense from both perspectives.”
Mensch: A team of five is faster than fifty unless uncoupled
“A team of five is faster than a team of 50. Except if you organize the team of 50 to be 10 teams of five that are sufficiently uncoupled.”
Mensch: DeepMind's initial Gemini development was too slow before recovering
“Gemini was a bit too slow, and I think they recovered sufficiently well since.”
Mensch: AI models will become merely starting points for developers
“And so I think that's the end state is models are effectively going to be A starting point for any AI application developer. They need to be surrounded by tools, by lifecycle management platform basically, and that's the one thing that we started to build.”
Mensch: AI application differentiation comes from data and user feedback
“Like general purpose models are a bit undifferentiated, but the differentiation that you need to create for your application comes from the data you put into it, the user feedback that you gather and the intelligence that you have to figure out what the applic…”