Falcon: Scaling massive AI models compensates for gaps in scientific understanding
William Falcon · Build and Deploy AI with Pytorch | Lightning AI Founder & CEO, William Falcon · May 1, 2023 · at 21:48
William Falcon is the Founder and CEO of Lightning AI. He is discussing the reliance on massive compute versus mathematical breakthroughs in AI model design.
0:00 / 0:15exact quote · 15.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I kind of think that the whole large scale model thing, It's really a gap in our scientific understanding of these models. Instead of looking at the math and coming up with a better loss function, we figured out that we could throw a compute at it, and get us pretty good results.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from William Falcon
Falcon: Open-source AI will quickly catch up to proprietary models
“So I think all, all roads lead to open source at the end of the day. No matter what you do or how much money you have, you cannot compete with the world's resources put together to do something. So some companies will have an edge for a bit, yes, but open sour…”
Falcon: Mathematical advances will eventually make future AI models small
“I think the math field will catch up, and we'll have some fancy regularizer that will make all this go away, and now we're back to, like, a small model. So I guess, I think that the future is small, large language models, I guess, that are specialized”
Falcon: There will not be a god-like AGI
“I don't think there's gonna be like a god AI roaming around.”
Falcon: Lightning AI will not offer hosted model APIs
“But no, we're not gonna be offering like services and APIs and all of that. And really that's so that we can have the same incentives as the open source community.”
Falcon: Solving AI hallucination will take roughly five years
“So I think it's still going to be, I don't know, like five years, probably like, I don't think it's going to be like one day suddenly it's solved. I think it's going to unlock industries sequentially. So like, year one, maybe we can do this industry, and then …”
Falcon: Lightning AI rebuilt Meta's LLaMA from scratch under Apache 2.0
“So we took the Lama, like, paper and implemented from scratch completely in Apache two, and we open sourced it. So it's fully, fully usable for enterprises, but we also gave you the training code, not just inference, and we also gave you fine tuning methods as…”