Morin: Interconnect dependency is the core difference between training and inference
Steeve Morin · Steeve Morin: Why Google Will Win the AI Arms Race & OpenAI Will Not | E1262 · 20VC with Harry Stebbings · Feb 24, 2025 · at 17:58
Steeve Morin, founder and CEO of ZML and former VP of Engineering at Zenly, discusses the architectural contrasts between AI training and production inference.
0:00 / 0:30exact quote · 30.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“In terms of infra, probably the number one thing that is the number one difference between these two is the need for interconnect. So if you do, you know, production, you, if you can avoid to have interconnect between, you know, let's say a cluster of GPUs, of course you will not go, you will, you know, avoid that, right? If you can. And this is why models have the sizes they have is so that people can run them without the need to connect multiple machines together.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Steeve Morin
Morin: Nvidia is far from the most efficient AI hardware platform
“But it's by far not the most efficient platform. And arguably, even in terms of software, it's not the best software platform.”
Morin: The market bubble around Nvidia H100 GPUs will burst
“There's going to be a need for inference. Very hard to say whether it will be worth, you know, everybody's money to do it on H 100. That is a bubble that I think will blow some time.”
Morin: GPUs are a clever workaround, not natively built for AI
“GPUs are, you know, are a good trick for AI, but they're not built for AI.”
Morin: AI chip oversupply will lead to GPUs selling at 30% value
“I very much worry there will be an oversupply of these chips. The problem is, is that, you know, remember, the chips are the collateral. So, you know, somewhere, you know, in the US or whatever, there's going to be a data center with like a thousand GPUs that …”