Shazeer: Text Is a Thousand Times More Information-Dense Than Images
Noam Shazeer · No Priors Ep. 46 | Best of 2023 with Sarah Guo and Elad Gil · Jan 11, 2024 · at 13:52
Noam Shazeer was not on this episode. A recording of them was played into it, so these are their words but not an appearance on No Priors. It still counts as said, and it is kept out of every score on their page.
Noam Shazeer, co-founder of Character AI and former Google Brain researcher, explains his rationale for focusing on text models over computer vision.
0:00 / 0:05exact quote · 5.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“An image is worth a thousand words, but it's like a million pixels. So like the text is like a thousand times as dense.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Noam Shazeer
Shazeer: Most Core AI Intelligence Will Come From Text Models
“But I think that a lot of the core intelligence is going to come from a From these text models.”
Shazeer Calls Next-Token Prediction an AI-Complete Problem
“And like, the problem is super simple to define. It's just like predict the next word, the fat cat sat on the, you know, like, okay, well, you know, what comes next? Like it's extremely easy to define. And if you can do a great job of it, like, you know, then …”
Core AI Intelligence Will Primarily Emerge From Text Models
“But I think that a lot of the core intelligence is going to come from these text models.”
AI Scaling Has Not Hit a Wall and Will Keep Improving
“So, and at the same time, like, I don't think anyone's seen a wall in terms of how good this stuff is. So I think it will just, it's just gonna keep getting better. I don't know what stops it.”
AI Models Can Generate Synthetic Data to Avoid Looming Shortages
“I think I'm not that worried about Coming up with data and I feel like we could probably like just generate some more with the AI.”
AI Assistants With a Single Neutral Corporate Persona End Up Boring
“Like if you're like such a public, you know, trying to present like one public persona that everyone likes, you're going to end up just being boring essentially. And people just don't want boring. You know, people want like the, you know want to interact with …”