Polosukhin: Transformers will be very hard for alternative architectures to match
Illia Polosukhin · No Priors Ep. 32 | With NEAR’s Illia Polosukhin · Sep 14, 2023 · at 34:48
Illia Polosukhin, co-author of the landmark 'Attention Is All You Need' paper and co-founder of NEAR Protocol, evaluates whether competing model architectures can overtake transformers.
0:00 / 0:08exact quote · 8.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The simplicity of this architecture and like, indeed, like the amount of optimization that's going into this right now is just, it will be really hard to match”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Illia Polosukhin
Polosukhin: AI agent CEOs will run organizations and replace middle management
“I really think one of the most interesting cases is organizations that are run completely by AI, right? Where quote unquote CEO role is taken by AI agent. Who's tasked by, you know, by community or board of directors or whatever is oversight governance is to, …”
Polosukhin: Decentralized AI training is unrealistic due to GPU bandwidth requirements
“And the reality right now that the requirements on bandwidth, right? Like people who are training these models right now, they have like a, you know, 800 gigabit connect right between the GPUs, right? So Maybe you have a hundred megabits on between this, usual…”
Polosukhin: Web3 and AI will start replacing traditional SaaS software
“So I think a lot of between web three and AI, a lot of SaaS will actually start being replaced because right now what SaaS is, is like one database. With a specific UI for a specific problem.”
Polosukhin: First organizations run by AI agents will emerge in 2023
“I think we'll see, you know, first organizations like this, probably even this year where Potentially with a simpler mission, some kind of more straightforward like KPI metrics, but where kind of this information propagation and onboarding of people happens al…”
Polosukhin: DAOs will be the first place AI managerial agents emerge
“I think DAOs is, and especially what happened with DAOs, there was a lot of people who were really excited about DAOs kind of as a concept. And so they put a lot of time running them, but it's actually a very like not interesting job, right? It's like you onbo…”
Polosukhin: Society needs human alignment rather than AI alignment
“So I have this view that we need human alignment instead of AI alignment. So right now, kind of when we talk about, you know, hey, we need to align AIs with like human values, but the reality is that, you know, all the problems that exist, they all exist becau…”