Text Models
topic on 5 shows · 7 statements across 6 episodes
Latent Space 
Lenny's Podcast 
No Priors 
the MAD Podcast 
the a16z Podcast
7 statements about Text Models, every show
Zeghidour: End-to-end speech models have massive switching costs for upgrades
“One drawback of speech-to-speech models is that since everything is integrated, when you go from a text model to the speech-to-speech model, you need to fine-tune it on speech data. So now it's the cost to switch the underlying text model is extremely high bec…”
Sherman Wu: Smaller size of image models drives faster iteration and proliferation
“Image models tend to be way smaller and like you can iterate on it a lot faster. Like that's why you get that crazy cool proliferation of like the image model side.”
Sherman Wu: Heavy compute for text model post-training bottlenecks verticalization
“For the text models, there's always going to be this like really big fat free training step that like you have to invest in here. And then even the post training side is like, You know, it's not the, it's not like the easiest thing. Like it's, you know we all,…”
Shulman: Understanding of music scaling laws is far behind text
“I don't think we know these things nearly as well as they're known in text. We have some notions of some of the scaling laws here, but I think yeah, we're just so, so far behind.”
Future LLMs will automatically expand short user prompts into detailed descriptions
“This will happen with text models. You can imagine a world where you go into ChatGPT and you say, write me a blog post about AI. It automatically will go and be like, let me generate a much higher fidelity description of what this person really wants, which is…”
Shazeer: Most Core AI Intelligence Will Come From Text Models
“But I think that a lot of the core intelligence is going to come from a From these text models.”