Opinion
Ross: West Coast VCs act like lemmings, East Coast VCs think independently
“Well, the way I like to put it is typical, typical, not all, but typical West Coast VCs are more like Lemmings, and typical East Coast VCs all think that they're smarter than each other.”
Insight
Ross: Replace one-on-ones and private emails with open communication to eliminate politics
“And so when you are leading groups of people, if you want to reduce the amount of politics and people going off and, you know, forming side cliques and all this, stop having one-on-ones, have big meetings with everyone who you want to tell something and tell t…”
Insight
Ross: Putting excess capital into startups no longer provides an advantage
“What's changed though is that unlike the Keynesian beauty contest where the winner is the one with the most money, in reality, there's a point at which you get enough money and you don't need more. And for the first time in history, the startups are not starve…”
Insight
Ross: Splitting LLM pre-fill and generation across different chips is a mistake
“What we realized was, and this is what most people get wrong when they're trying to do this themselves. They'll, they'll take the reading of tokens or what's called pre-fill, and they'll do that on one piece of hardware, and then they'll put the generation of …”
Insight
Ross: Faster inference compute directly makes AI models smarter
“Now bring in the LPU, and you can go deeper, faster, you can search faster, and so you can actually make a model smarter by making it faster. And so the realization is, and now that you've got this ability to reflect, think deeply, and change the outcome based…”
Insight
Ross: Hiring requires vetting negatives rather than searching for positive traits
“The biggest flip in my hiring was when I went from looking for positives, which is what you do when you're trying to grow talent, to looking for negatives, which is what you do when you're trying to select talent.”
Assertion Not checkable as stated
Ross: Marginal cost of software code is approaching zero
“What I'm seeing is that code is becoming almost free. It's, it, the marginal cost is approaching zero.”
Assertion Not checkable as stated
Ross: Jensen Huang never holds private one-on-one meetings at NVIDIA
“There is no circumstance at NVIDIA where there, where Jensen has one-on-ones with people and tells them one thing.”
Insight
Ross: GPUs excel at LLM attention while LPUs excel at applying weights
“The LPU and GPU, as mentioned, they, they're better at different parts of what's, what's called the decoder layer of an LLM. The GPU is better at the attention portion, and the LPU is better at sort of applying the weights, which is the thing that gets trained…”
Assertion Supported
Ross: Porting AlphaGo to TPUs jumped its ELO by hundreds of points
“AlphaGo running on GPUs had an ELO score of about 3200. Lease it all was about 35, 35 50. And I might be getting the first digit wrong. It might be 2000 instead of 3000. But like, you know, it was more than 200. And then when put on on TPUs, it actually went t…”
Assertion Not checkable as stated
Ross: AlphaGo running on GPUs never found Move 37
“In the second game, there was this famous move called move 37, which was creative. It was original. It actually wasn't completely original. It was a one in 10,000 game move. It had been in the canon of games that we're trained on. But when we went back and pla…”
Opinion
Ross: GPUs are now better than Google TPUs
“So over time, these GPUs have, you know, gotten, you know, as good and better than TPUs, you know, as the career of the TPU, I have to admit, GPUs are now better.”
Assertion Not checkable as stated
Ross: GitHub CEO asked Groq for LPUs amid GPU shortages
“When LLMs first started to become a thing, I remember getting a phone call from the CEO of GitHub basically saying, I need a bunch of GPUs. We've now gotten LLMs to be able to do code completion, even though, you know, we're part of Microsoft and everything yo…”
Assertion Not checkable as stated
Ross: 80% of Groq employees took salary cuts to statutory minimums
“80% of the employees participated. And I think about half went to the statutory minimum salary by law. And remember, engineers get paid hundreds of thousands of dollars. These folks like cut their salary down to like 50, 60,000 dollars, whatever the statutory …”
Assertion Not checkable as stated
Ross: Groq's NVIDIA deal closed within three weeks of first call
“The call where the idea was first floated was about three weeks before money was in the bank.”
Insight
Ross: Combining GPUs and LPUs yields superior performance across curves
“And so GPUs and LPUs combined ended up giving better performance across the performance curves.”
Disclosure
Ross: Groq initially approached Jensen Huang to buy 100,000 GPUs
“We had implemented it and we had gone to Jensen asking, could we buy about a 100,000 GPUs because we were going to deploy them ourselves.”
Insight
Ross: Agentic AI drives exponential growth by delegating tasks to other AIs
“AI is really good at using AI. And that's really what agentic is, right? So humans benefit from using AI. So does AI. Just like you would do research about me before I show up on your show, AI is going to kick off a job doing research on different tools it's g…”
Prediction Not checkable as stated
Ross: Machine micropayments will cause autonomous payment volumes to skyrocket
“Yeah, I think it's still early, and one of the limiters is payments aren't really built for this yet, but if you can make micropayments, the number of payments is going to skyrocket.”
Insight
Ross: Workers Are Shifting from Individual Contributors to Leaders of AI
“Yeah, and this also goes towards a shift of going, people are moving from being ICs or individual contributors to all being leaders, but leaders of AI.”
Disclosure
Ross: Groq hires highly autonomous people who would fail in corporate setups
“When I hire people, I hire very autonomous people who often go off and execute on their own, and they would be terrible in most corporate environments.”
Assertion Not checkable as stated
Ross: Learning to manage people cost Groq three to four years
“The first thing that you have to do as a founder is you have to go from the technical thing that you know how to do and that you can add value with to learning how to manage people. And for me, that probably cost Grok three to four years.”
Assertion Not checkable as stated
Ross: Groq Faced High Turnover Over Skepticism About Fast Inference
“There was a lot of pushback on, like, we had a lot of turnover at this point. A lot of people were leaving. This was probably three or four years ago. And so a lot of people were leaving, saying there was no point to fast inference. It wasn't going to add any …”
Insight
Ross: Stating 'I intend to' prevents pessimism and filters feedback
“And the basic idea in intentional leadership is if I ask someone, should I do something? Oh, they have opinions. Most people will be pessimistic and give you negative opinions. On the other hand, if you express intentional leadership, you say, I intend to do t…”