Doshi: Curated data in fine-tuning matters more than vision algorithmic tricks
Suhail Doshi · No Priors Ep. 60 | With Playground AI Founder Suhail Doshi · Apr 18, 2024 · at 9:13
Playground AI founder Suhail Doshi discusses the technical factors that drive major performance improvements in generative vision models.
0:00 / 0:13exact quote · 13.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I think there are all these, there are like lots of tricks that sometimes get you like 10, 20, sometimes two X improvements. But I think the number one trick is like really just like that last phase of You know, a supervised fine tune where you're finding like really great curated data.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Suhail Doshi
Doshi: Most AI benchmarks are flawed and optimized for marketing
“I think that most evals in the industry are relatively flawed. Like a lot of them are doing like benchmarks on things that maybe are valuable from the purposes of marketing, but are not necessarily well correlated with the With what maybe users care about.”
Doshi: Vision AI can access infinite real-world data unlike internet text
“Whereas like with vision, at least you can like make a robot that just like travels down the street and like just keeps taking pictures of everything. You can get like infinite training data with vision, but it might be trickier to like sort of filter and clea…”
Doshi: Playground AI is probably number two in text-to-art
“Maybe like probably like number two, I suspect, I guess at like text to art at the moment, just because we're training these models from scratch, we're getting, we're closing the gap really rapidly as rapidly as we can around all the various like kind of use c…”
Doshi: 3D software tools tend not to make much money
“One issue with three D is that it tends to be better to work on three D. If you're like making the content, like you're making Pixar movies the tools in three D tend to not like make as much money.”
Doshi: Diffusion Transformers Alone Lack Utility Without Integrated Language Models
“Transformers are definitely, I think transformers are definitely like the right direction, but I don't think that we're going to get a lot of you enough utility if we're not like somewhat trying to figure out a way to combine the great, amazing knowledge of li…”