Large Language Models Are Creative Fiction Engines Rather Than Factual Databases
Emad Mostaque · No Priors Ep. 3 | With Stability AI’s Emad Mostaque · May 3, 2023 · at 36:14
Stability AI founder Emad Mostaque explains the architectural limits of LLM compression during a discussion on AI reliability and alignment.
0:00 / 0:19exact quote · 19.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I do say these large models as well should be viewed as fiction, creative models, not fact models, because otherwise we've created the most efficient compression in the world. Does it make sense you can take terabytes of data and compress it down to a few gigabytes with no loss? No, of course you lose something. You lose the factualness of them, but you keep the principle-based analysis of them.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Emad Mostaque
Stability AI Has More Compute Available Than Microsoft or Google
“The reality is we have more compute available to us than Microsoft or Google.”
All Base Foundation Models Will Ultimately Open Source Their Pre-Training Phase
“So my take is that foundation models will all be open source for the deep learning phase.”
Emerging Market AI Systems Will Outperform the West Within Five Years
“I am certain that system will outperform anything in the West within five years.”
Only About 100 People Worldwide Possess Skills to Train Foundation Models
“There's only like a hundred people who can train models in the world. Like it's not really a science. It's more an art, like losses explode all over the place when you try to do something.”