Sivulka: Fine-tuned models will never catch up to inference scaling
George Sivulka · George Sivulka, Co-Founder & CEO @Hebbia: The Future of Foundation Models | E1250 · 20VC with Harry Stebbings · Jan 22, 2025 · at 39:23
George Sivulka, founder and CEO of Hebbia, discusses AI model scaling strategies with host Harry Stebbings.
0:00 / 0:27exact quote · 27.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And so you saw the idea of, ah, of post training or kind of like this refined, verticalized model creation, ah, just always would lose to scaling laws. And maybe we're at the end of scaling laws at training, but I actually think, you know, Hebbia and now OpenAI and a variety of other companies are starting to pioneer the idea of scaling laws and inference, and I actually think that it will, like, nothing that, that other players can do to fine tune models will ever catch up.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from George Sivulka
Sivulka: Retrieval Augmented Generation (RAG) does not work at all
“That's a bit of a plot twist over here. We, I actually don't think RAG works at all.”
Sivulka: Great founders have messed up childhoods, are gay, or adopted
“You can bucket great founders into three backgrounds. The most common is that you have kind of a messed up childhood. The second most common would be, ah, you're gay, and the third most common would be you were adopted. Look at like a list of all-time greats. …”
Sivulka: ChatGPT is a poor product that acts like a calculator
“I don't even think ChatGPT is a really good product. It's like a calculator. It's got the technology in there encapsulated in a very simple form. But it's not a product that, like in Excel, that lets you just build whatever you'd like with it.”
Sivulka: 90% of enterprise AI is vapor and usage stats are fake
“I think, like, 90% of enterprise AI right now is almost like this vapor where, hey, we swear it works, like, look at this amazing demo where we ask, What does the CEO say about the investment? And the minute that they actually go to, you know, try to use it in…”
Sivulka: Satya Nadella is wrong about business apps collapsing into agents
“I actually don't agree with that at all. I think he's completely wrong. I think it depends on how you define a business app. But I actually think that if the new business apps are platforms, you'll, you'll actually start to see those platforms really take hold…”