Polosukhin: Market full of Transformer-optimized accelerators will launch by 2024
Illia Polosukhin · No Priors Ep. 32 | With NEAR’s Illia Polosukhin · Sep 14, 2023 · at 41:30
NEAR Protocol co-founder and Transformer co-author Illia Polosukhin discusses custom silicon efforts by major tech companies.
0:00 / 0:11exact quote · 11.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And so like, we're going to have a, you know, a market full of hardware accelerators, which are still optimized for transformers, or at least like similar structured architectures hitting the market like this year and next year.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Illia Polosukhin
Polosukhin: AI agent CEOs will run organizations and replace middle management
“I really think one of the most interesting cases is organizations that are run completely by AI, right? Where quote unquote CEO role is taken by AI agent. Who's tasked by, you know, by community or board of directors or whatever is oversight governance is to, …”
Polosukhin: Decentralized AI training is unrealistic due to GPU bandwidth requirements
“And the reality right now that the requirements on bandwidth, right? Like people who are training these models right now, they have like a, you know, 800 gigabit connect right between the GPUs, right? So Maybe you have a hundred megabits on between this, usual…”
Polosukhin: Web3 and AI will start replacing traditional SaaS software
“So I think a lot of between web three and AI, a lot of SaaS will actually start being replaced because right now what SaaS is, is like one database. With a specific UI for a specific problem.”
Polosukhin: Transformers will be very hard for alternative architectures to match
“The simplicity of this architecture and like, indeed, like the amount of optimization that's going into this right now is just, it will be really hard to match”
Polosukhin: First organizations run by AI agents will emerge in 2023
“I think we'll see, you know, first organizations like this, probably even this year where Potentially with a simpler mission, some kind of more straightforward like KPI metrics, but where kind of this information propagation and onboarding of people happens al…”
Polosukhin: DAOs will be the first place AI managerial agents emerge
“I think DAOs is, and especially what happened with DAOs, there was a lot of people who were really excited about DAOs kind of as a concept. And so they put a lot of time running them, but it's actually a very like not interesting job, right? It's like you onbo…”