AWS VP: LLM architectures will shift to hierarchical edge-and-cloud inference
Swami Sivasubramanian · AWS VP of AI and Data Swami Sivasubramanian — GenAI's Growth Potential · May 1, 2024 · at 31:30
Swami Sivasubramanian is the VP of AI and Data at AWS. He discusses how on-device AI from companies like Apple interacts with cloud computing demand.
0:00 / 0:41exact quote · 41.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So, I expect LLNs to evolve in the same way, where there are a lot of, ah, decisions, especially if you have a powerful computing device, ah, which, ah, many smartphones and others, ah, to be having that, You can actually run some of those, ah, simple LLM things at the edge and dual stamp, but there is still going to be a huge amount of, ah, inference where you need more data that is available, where are more capabilities and so forth. So I expect this hierarchical capability and hierarchical inference to happen in the same way how it played out in the deep learning era as well.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Swami Sivasubramanian
AWS's Sivasubramanian: AI scaling is not close to hitting a wall
“One, I don't think we are anywhere close to yet hitting that wall yet. So I do think we are going to actually see a lot of net new innovations when it comes to optimizing how to actually train these models in parallel and get better Utilization so that we can …”
AWS AI Chief: AI architectures beyond pure transformers are absolutely necessary
“I actually think, ah, new architectures are absolutely necessary in the future, ah, and, ah, there are already hybrid architectures evolving. I mean, ah, I mean in terms of state space models to actually connectors hybrid models between state space to transfor…”
AWS AI Chief: Enterprise AI needs smaller distilled models, not monolithic LLMs
“To put these LLMs to work to solve real world problems, you got to actually take some of these models and then customize it, and the end result is not the biggest model. It is actually a much more customized, smaller model or a distal model to solve specific b…”
AWS VP: Pure LLMs Won't Deliver Effective AI Reasoning Without Other Tools
“Reasoning capability is not, if we view it as purely like everything is going to be LLM driven, we as industry are going to be very, very not satisfied. That's why it has to be actually more iterative. And we work with LLM as one of the ingredients. It is not …”
Sivasubramanian: Trainium2 offers 4x faster training and 2x energy efficiency
“AWS actually, for instance, we have invested in technologies such as Trinium-II chips, which actually can deliver up to four X faster training. And, ah, also it is while improving energy efficiency up to two times, as an example.”