Weinberg: Intuition for AI progress comes from hammering models through failures
Winston Weinberg · No Priors Ep. 101 | With Harvey CEO and Co-Founder Winston Weinberg · Feb 14, 2025 · at 6:00
Harvey CEO Winston Weinberg explains to Sarah Guo how he developed conviction around model capability jumps between GPT-3 and GPT-4.
0:00 / 0:19exact quote · 19.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“It seems like there was this large gap between I log into ChatGPT or I just use GPT-IV from an API or whatever it is, and I try something a couple times versus I'm going to sit there and just hammer on this until it works, right? and I think if you do that enough, you get the intuition for where things are going and how much better they can get.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Winston Weinberg
Weinberg: Standard AI benchmarks are useless for evaluating legal AI
“Most benchmarks are completely useless for us, right? And so we'll get a model, you know, someone will give us early access to a model and they'll say it's way better on all of these benchmarks and we'll respond. It actually isn't like, it's not used as useful…”
Weinberg: Automated legal tasks will shift to fixed-fee pricing
“I don't think the billable hour is going to just completely disappear. I think what is going to happen is a lot of these tasks that are going to get, you know, AI can automate a lot of that task with a lawyer in the loop. Those tasks will end up being kind of …”
Weinberg: AI founders must aim to transform entire industries
“If you are using these tools and you don't think that you can take basically AI and apply to X industry and transform the entire industry, I don't think you're thinking ambitiously enough, right?”
Weinberg: AI platforms must expand modular workflows and collapse UI complexity
“These models can't just one shot all of these really complex legal tasks or in these other domains like tax and provide other professional services. And so what you have to do is you have to build a platform that is kind of constantly expanding and constantly …”
Weinberg: Enterprise AI requires pairing broad productivity tools with chained vertical workflows
“You need to build productivity tools, and what I mean by this is
Things that are useful for the highest amount of seats, right?
And then you also need to build things that are streamlined vertical workflow from start to finish, right?
And what you can do is…”
Weinberg: Productivity AI needs lower quality thresholds if work is verifiable
“On the productivity side, the minimum viable quality of that output can be lower because you're selling seats, And at the end of the day, there are multiple people reviewing it, right? And so what you want to do in that state is have just show your work, right…”