AWS AI Chief: AI architectures beyond pure transformers are absolutely necessary
Swami Sivasubramanian · AWS VP of AI and Data Swami Sivasubramanian — GenAI's Growth Potential · May 1, 2024 · at 9:58
Swami Sivasubramanian, AWS VP of AI and Data, discusses whether scaling LLMs through traditional data and compute is hitting a wall.
0:00 / 0:19exact quote · 19.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I actually think, ah, new architectures are absolutely necessary in the future, ah, and, ah, there are already hybrid architectures evolving. I mean, ah, I mean in terms of state space models to actually connectors hybrid models between state space to transformers and so forth.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Swami Sivasubramanian
AWS's Sivasubramanian: AI scaling is not close to hitting a wall
“One, I don't think we are anywhere close to yet hitting that wall yet. So I do think we are going to actually see a lot of net new innovations when it comes to optimizing how to actually train these models in parallel and get better Utilization so that we can …”
AWS AI Chief: Enterprise AI needs smaller distilled models, not monolithic LLMs
“To put these LLMs to work to solve real world problems, you got to actually take some of these models and then customize it, and the end result is not the biggest model. It is actually a much more customized, smaller model or a distal model to solve specific b…”
AWS VP: Pure LLMs Won't Deliver Effective AI Reasoning Without Other Tools
“Reasoning capability is not, if we view it as purely like everything is going to be LLM driven, we as industry are going to be very, very not satisfied. That's why it has to be actually more iterative. And we work with LLM as one of the ingredients. It is not …”
Sivasubramanian: Trainium2 offers 4x faster training and 2x energy efficiency
“AWS actually, for instance, we have invested in technologies such as Trinium-II chips, which actually can deliver up to four X faster training. And, ah, also it is while improving energy efficiency up to two times, as an example.”