Ross: Groq Faced High Turnover Over Skepticism About Fast Inference
Jonathan Ross · Why Asking the Right Questions Is the Most Important Skill in the AI Age · Jul 5, 2026 · at 41:37
Groq founder Jonathan Ross explains internal resistance and skepticism toward the necessity of ultra-fast inference speed.
0:00 / 0:15exact quote · 15.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“There was a lot of pushback on, like, we had a lot of turnover at this point. A lot of people were leaving. This was probably three or four years ago. And so a lot of people were leaving, saying there was no point to fast inference. It wasn't going to add any value to the ecosystem.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jonathan Ross
Ross: West Coast VCs act like lemmings, East Coast VCs think independently
“Well, the way I like to put it is typical, typical, not all, but typical West Coast VCs are more like Lemmings, and typical East Coast VCs all think that they're smarter than each other.”
Ross: Replace one-on-ones and private emails with open communication to eliminate politics
“And so when you are leading groups of people, if you want to reduce the amount of politics and people going off and, you know, forming side cliques and all this, stop having one-on-ones, have big meetings with everyone who you want to tell something and tell t…”
Ross: Putting excess capital into startups no longer provides an advantage
“What's changed though is that unlike the Keynesian beauty contest where the winner is the one with the most money, in reality, there's a point at which you get enough money and you don't need more. And for the first time in history, the startups are not starve…”
Ross: Splitting LLM pre-fill and generation across different chips is a mistake
“What we realized was, and this is what most people get wrong when they're trying to do this themselves. They'll, they'll take the reading of tokens or what's called pre-fill, and they'll do that on one piece of hardware, and then they'll put the generation of …”
Ross: Faster inference compute directly makes AI models smarter
“Now bring in the LPU, and you can go deeper, faster, you can search faster, and so you can actually make a model smarter by making it faster. And so the realization is, and now that you've got this ability to reflect, think deeply, and change the outcome based…”