Sep 14, 2023 · 42m · no-priors
No Priors Ep. 32 | With NEAR’s Illia Polosukhin
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
NEAR Protocol co-founder and Transformer co-author Illia Polosukhin joins No Priors to discuss the intersection of artificial intelligence and Web3. The conversation explores autonomous AI agents with economic agency, cryptographic provenance for combating deepfakes, decentralized compute and data marketplaces, and the architectural evolution of foundation models.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 22.5% of the talking time here. How this is scored →
The hosts as informed peer 5.3 Guest teaching 4.7 Guest disagreement 0.9 The hosts pushing back 0.6
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 10:07 Reframing AI alignment as human alignment
Illia directly rejects the conventional framing around AI alignment, arguing the core challenge is human misinformation and coordination failures rather than model alignment.
Hardest push from the hosts ▶ 6:44 Challenging blockchain in biotech researchSarah directly questions why a traditional commercial cancer research entity would need blockchain and AI over standard coordination tools.
Biggest teaching moment ▶ 22:22 Technical realities of decentralized training vs inferenceIllia breaks down why crypto mining GPUs and decentralized nodes cannot train LLMs due to 800Gbps interconnect requirements, clarifying that decentralized compute is viable only for inference.
The host holds their own ▶ 39:34 Elad's historical Wintel monopoly comparisonElad demonstrates high-level industry synthesis by connecting the mutual optimization loop of GPUs and Transformers to the Wintel architectural lock-in of the 1990s.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Origins of the Landmark Attention and Transformers Paper | 4 | 5 | 0 | 0 | Sarah sets up the historical context of the Attention paper and Near's origins. Illia explains how early work on NLU and question answering led to attention mechanisms and how crowd-sourcing developer compensation issues prompted Near's pivot to blockchain. | |
| Defining NEAR as a Blockchain Operating System | 5 | 4 | 1 | 0 | Elad demonstrates knowledge of Near's founding history and core technical talent while asking where AI and Web3 intersect. Illia details how autonomous AI agents can act as economic actors using crypto accounts to coordinate and manage organizations. | |
| Real-World Use Case: AI Coordination in Biotech Research | 5 | 3 | 0 | 1 | Sarah challenges the necessity of blockchain for biotech cancer research coordination. Illia explains how decentralized grant funding, lab task allocation, and automated performance accountability can eliminate human overhead and bias. | |
| Human Alignment, Misinformation, and Cryptographic Content Provenance | 4 | 6 | 2 | 0 | Illia reframes the standard alignment debate, asserting that alignment is fundamentally a human problem rather than an AI problem. He articulates how cryptographic provenance and web of trust models analogous to SSL can combat personalized misinformation. | |
| Blockchain Identity, Account Permissions, and the SSL Transition Model | 6 | 4 | 1 | 1 | Elad probes why identity primitives on blockchain have lagged despite smart contracts providing execution without intelligence. Illia differentiates raw cryptographic keys from readable named accounts and permission delegation on Near. | |
| AI Threats: Hyper-Personalized Propaganda and Voice Impersonation | 5 | 4 | 1 | 0 | Sarah and Elad explore system failure modes, with Elad pointing to voice cloning and financial social engineering tools. Illia explains why existing hardware secure enclaves must be combined with cryptographic signing standards to safeguard communication channels. | |
| Decentralized Compute: Decentralized Inference versus Training Constraints | 6 | 6 | 2 | 0 | Elad asks whether repurposed crypto mining GPUs or decentralized networks can handle model training. Illia corrects the premise by highlighting interconnect bandwidth limits (800Gbps needed for training clusters vs consumer internet) while arguing decentralized compute makes sense for private inference. | |
| Decentralized Data Labeling, Game Theory, and Quality Control | 6 | 5 | 1 | 2 | Sarah raises the practical hurdle of RLHF quality control that forces frontier labs to keep labeling in-house. Illia outlines how economic game theory, honeypots, and staking mechanisms create better self-evaluation and quality guarantees on open platforms. | |
| NEAR Ecosystem Developments and Reimagining SaaS Architecture | 4 | 5 | 0 | 0 | Illia shares Near ecosystem updates and shares his vision for the unbundling of enterprise SaaS into open user-owned databases paired with custom generative frontends. | |
| Dynamic Generative UIs and Agent-Driven Workflows | 5 | 6 | 1 | 0 | Elad and Sarah discuss dynamic interfaces, and Sarah asks whether future AI advances will be bigger Transformers or new architectures. Illia proposes allowing models 'thinking time' via blank compute tokens before emitting output tokens. | |
| Hardware-Software Lock-In: The Wintel Analogy for Transformers | 8 | 4 | 1 | 2 | Elad draws a sophisticated parallel between the hardware-software lock-in of modern GPUs/Transformers and the 1990s Wintel duopoly. Sarah adds nuance regarding supply shortages incentivizing heterogeneous hardware, which Illia counters by noting most new hardware accelerators still target Transformer-like operations. |
Statements from this episode (23)
Polosukhin: LSTMs Were Too Slow for Production as Documents Scaled
“The state of the art at this time was LSTMs, Recurring Neural Networks, which you could not launch in production at all because they're too slow and take a fair bit of time to process as documents scale.”
Polosukhin: NEAR pivoted to blockchain after payment issues with AI data labelers
“We started looking into blockchain just like to solve our own problem. The AI kind of exponential explosion didn't happen at the time. And so we saw an opportunity of, we can actually build a blockchain that we would use to solve this first and focus on that w…”
Polosukhin: Blockchain accounts turn AI agents into paying economic actors
“If we, you equip them with a blockchain account, right? They are now becoming an economic agent that is able to pay other people and pay other AIs to do work, right?”
Polosukhin: AI agent CEOs will run organizations and replace middle management
“I really think one of the most interesting cases is organizations that are run completely by AI, right? Where quote unquote CEO role is taken by AI agent. Who's tasked by, you know, by community or board of directors or whatever is oversight governance is to, …”
Polosukhin: First organizations run by AI agents will emerge in 2023
“I think we'll see, you know, first organizations like this, probably even this year where Potentially with a simpler mission, some kind of more straightforward like KPI metrics, but where kind of this information propagation and onboarding of people happens al…”
Polosukhin: DAOs will be the first place AI managerial agents emerge
“I think DAOs is, and especially what happened with DAOs, there was a lot of people who were really excited about DAOs kind of as a concept. And so they put a lot of time running them, but it's actually a very like not interesting job, right? It's like you onbo…”
Polosukhin: Society needs human alignment rather than AI alignment
“So I have this view that we need human alignment instead of AI alignment. So right now, kind of when we talk about, you know, hey, we need to align AIs with like human values, but the reality is that, you know, all the problems that exist, they all exist becau…”
Polosukhin: Platforms will adopt SSL-style cryptographic locks for content verification
“I think one of the imported pieces will be kind of a green lock, similar to SSL transition on the content, right? Like as you go to YouTube, as you go to you know, New York times, you actually will see that like, Hey, this content been signed by this party and…”
Polosukhin: 2024 US election will see fake AI candidates and tailored agendas
“I think there will be probably next year will be very interesting in US because I think this will be a place where everybody will just take whatever their toys have in toolbox and do it even just for kicks, right? Even if it's not malicious, although some play…”
Polosukhin: Law enforcement lacks tools to identify AI-generated media
“The people are using these tools now in very malicious ways right now. And law enforcement don't have a, like really good ways to deal with this. And so I think everything from this, like on camera, like signing, we need this now, like they really have no way …”
Gil: 30 seconds of audio in voice APIs can fool banks
“This is where people would be using APIs like Element or LFG or 11 Labs to create a voice snippet, right, where they'll upload, to your .30 seconds of voice, train a model, and then the output sounds close enough to the person that you could fool a financial a…”
Polosukhin: Repurposed crypto mining GPUs cannot meet frontier AI training needs
“The challenges, the GPUs there are like, not the ones that AI folks want to use, right? Like kind of all the AI is really zeroed in on like, how do we get a 100 or H 100 and the GPUs that like folks used for Ethereum mining and like similar is like older ones …”
Polosukhin: Decentralized AI training is unrealistic due to GPU bandwidth requirements
“And the reality right now that the requirements on bandwidth, right? Like people who are training these models right now, they have like a, you know, 800 gigabit connect right between the GPUs, right? So Maybe you have a hundred megabits on between this, usual…”
Polosukhin: Inference demands vastly more aggregate compute than AI model training
“I think an inference is really interesting because we do need so much more compute for inference than we need for training, right? Like it's a very interesting like economy of scale. You train once, like Lama trained once and then everybody runs it everywhere.”
Polosukhin: Decentralized Web3 Marketplaces Will Be More Effective for AI Data Labeling
“I think decentralized kind of a web three marketplace is a more effective way to do this.”
Guo: Major AI Labs Insource Annotators Due to Vendor Quality Deficits
“One thing that I've seen with significant research labs is like still continued insourcing of annotators for both pre-training sets and LHF because some of the external services and marketplaces can't get to the level of quality that they're looking for in par…”
Polosukhin: Staked Buy-Ins Enforce Data Quality Better Than Employment Contracts
“There's an actual, like very clear, like economic game theory where people have buy-ins. They lose them if they like do poor quality of work. And so they have Like way more incentive to do this versus like, let's say if you're working on a contract, there's li…”
Polosukhin: Web3 and AI will start replacing traditional SaaS software
“So I think a lot of between web three and AI, a lot of SaaS will actually start being replaced because right now what SaaS is, is like one database. With a specific UI for a specific problem.”
Polosukhin: Future SaaS interfaces will be hybrid AI agents and dynamic UIs
“It will be a hybrid. So I like in my imagination right now, at least I expect like you can describe a business process, which is like, Hey, you know, when we have a new creative from like marketing department, Spin up a Twitter campaign and create me a dashboa…”
Polosukhin: Transformers will be very hard for alternative architectures to match
“The simplicity of this architecture and like, indeed, like the amount of optimization that's going into this right now is just, it will be really hard to match”
Polosukhin: Internal knowledge search beats external search at inference time
“The fact that this model is, like, doing a really effective search in kind of this knowledge space means that probably, like, pushing more into that concept is more useful than doing more searches at inference time because, like, it means you already lost all …”
Gil: GPU and Transformer lock-in is stronger than 1990s Wintel
“And so this is, I feel like, a stronger version of that in some sense, where you have the underlying compute architecture and the most important model reinforcing each other in a way that kind of locks both of them in.”
Polosukhin: Market full of Transformer-optimized accelerators will launch by 2024
“And so like, we're going to have a, you know, a market full of hardware accelerators, which are still optimized for transformers, or at least like similar structured architectures hitting the market like this year and next year.”