Mar 5, 2026 · 1h 16m · latent-space
Why Every Agent Needs a Box — Aaron Levie, Box
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of the Latent Space Pod, Box CEO Aaron Levie joins hosts Swyx and Jeff Huber to discuss why enterprise content infrastructure, governance, and context management are vital for autonomous AI agents. Levie analyzes the transition from AI coding tools to high-stakes knowledge work, explaining how organizational restructuring, sandboxed storage, and expanding software engineering volume will shape the enterprise AI landscape.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The hosts as informed peer 5.3 Guest teaching 5.1 Guest disagreement 1.7 The hosts pushing back 1.7
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 17:19 Levie rejects easy-mode workflow premise
Levie directly counters the notion that enterprise AI can avoid deep workflow re-engineering, pointing out that OpenAI and Anthropic have to deploy forward-deployed engineers to make it work.
Hardest push from the hosts ▶ 17:06 Swyx challenges enterprise consulting overheadSwyx pushes back directly against Levie's requirement for workflow restructuring, warning that customers will reject expensive consulting and prefer vendors that meet them where they are.
Biggest teaching moment ▶ 11:36 Levie's contrast of coding vs enterprise knowledgeLevie systematically enumerates the structural advantages AI coding enjoyed over general enterprise knowledge work, educating the hosts on why broader enterprise agent adoption lags.
The host holds their own ▶ 29:45 Huber cites Chroma context pruning researchHuber demonstrates strong technical domain authority by citing Chroma's experimental findings on agentic search failure modes and context window self-distraction.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Welcome to Latent Space: Introducing Aaron Levie | 4 | 5 | 1 | 1 | Swyx introduces Aaron Levie and co-host Jeff Huber, pitching the tagline 'Every agent needs a box.' Levie enthusiastically expands on how enterprise files transform into active data sources for autonomous agents. | |
| Agent Governance, Security, and Identity Architecture | 5 | 6 | 2 | 2 | Swyx questions how identity and security boundaries work between human accounts and autonomous agents. Levie explains the nuanced liability and permission model differences between human employees and autonomous agents. | |
| Rethinking RBAC and File Sharing for AI Workflows | 6 | 5 | 2 | 3 | Jeff Huber argues that traditional RBAC is obsolete for AI workloads. Levie details Box's waterfall permissions hierarchy while correcting Swyx's Fortune 500 customer percentage statistic. | |
| Why AI Coding Exploded First Versus Knowledge Work | 4 | 7 | 3 | 1 | Levie lays out a comprehensive breakdown of why coding succeeded first in AI adoption due to text-in/text-out uniformity and documentation versus messy enterprise knowledge work. | |
| Enterprise Workflow Re-engineering and Context Limits | 7 | 5 | 4 | 6 | Swyx pushes back on Levie's claim that enterprises must re-engineer workflows, warning that consultant-heavy approaches will lose to plug-and-play competitors. Levie counters by citing OpenAI and Anthropic deploying forward-deployed engineers. | |
| Context Engineering, Token Limits, and Search Judgments | 6 | 6 | 2 | 2 | The hosts and Levie discuss context rot and search termination problems, exploring how models must learn when to stop querying missing enterprise information. | |
| Code Slop Versus Enterprise Liability and Knowledge Risk | 5 | 6 | 2 | 1 | Levie points out that while code slop still yields functioning executables, enterprise document slop creates critical corporate liability and contractual risk. | |
| Context Pruning and Agentic Search Failure Modes | 7 | 5 | 1 | 1 | Huber shares Chroma's research on agentic search and context pruning. Levie discusses Box's private multi-industry evals showing frontier model reasoning improvements. | |
| Inside Box's Core AI Team and Infrastructure | 4 | 5 | 2 | 1 | Swyx prompts Levie to talk about Box's internal AI organization. Levie clarifies that AI is treated as an existential focus across a tight core team rather than a generic innovation lab. | |
| Read Versus Write Workflows for Enterprise Agents | 6 | 5 | 1 | 1 | Huber asks about write workflows versus read workflows. Levie outlines how agents will use Box as a shared file system for generating slides, memos, and workspace artifacts. | |
| Evaluating Companies as Markdown Skills Files | 6 | 5 | 2 | 2 | Swyx asks whether a company can simply be defined by markdown skills files. Huber points out tacit apprenticeship knowledge, and Levie explains the friction of rapidly changing real-world business contexts. | |
| Knowledge Graphs Versus Lindy Wikis and Emergent Memory | 6 | 4 | 1 | 2 | Swyx probes Levie on knowledge graphs. Levie demurs from ideological graph debates, while Huber and Swyx contrast emergent graph structures with persistent wiki-style markdown. | |
| Aaron Levie's Founder Mode and Existential AI Focus | 4 | 6 | 1 | 1 | Swyx asks Levie about his interpretation of 'founder mode.' Levie details delegating standard operations while retaining intense hands-on control over existential AI architecture choices. | |
| The Weekend Agent Compute Grind | 5 | 4 | 1 | 1 | The hosts discuss weekend agent compute utilization and Huber asks about Levie's production function and high-frequency public writing habits on Twitter. | |
| Cinematic Influences, A24, and AI in Filmmaking | 4 | 4 | 1 | 1 | Swyx and Levie discuss indie film influences, A24, and the nuances of generative AI media accelerating film previz without flooding feeds with low-quality slop. | |
| TechCrunch Backyard Parties and Ecosystem Catalysts | 5 | 4 | 1 | 1 | Swyx reflects on TechCrunch house parties that catalyzed Box's early funding, asking how modern AI ecosystem builders can replicate that community flywheel. | |
| DevRel Boom, Forward Deployed Engineers, and Engineering Value | 6 | 5 | 2 | 3 | Swyx and Levie debate DevRel supply shortages and whether forward-deployed engineering (FDE) is glorified consulting or an essential technical distribution vehicle. | |
| AI Multiplies Software: The Future of AI Engineering | 5 | 5 | 1 | 1 | The panel concludes on the long-term outlook for software engineering, agreeing that AI tooling dramatically expands code volume and reinforces the primacy of technical builders. |
Statements from this episode (35)
Levie: Enterprise file archives will become extremely valuable via AI agents
“All of that information becomes valuable to the enterprise and it's going to become extremely valuable to end users because now they can have agents go find what they're looking for and produce new New value and new data on that information. And it's going to …”
Levie: Enterprises will inevitably deploy an order of magnitude more agents than people
“You know, whether you think the number is 10 X or a hundred X or whatever the number is, we're going to have some order of magnitude more agents than people. That's inevitable. It has to happen.”
Levie: Prompt injection against AI agents will cause major enterprise security breaches
“There's going to be just incredibly spectacularly crazy security incidents that will happen with agents because you'll prompt inject an agent and Sort of find your way through the CRM system and pull out data that you shouldn't have access to.”
Levie: AI agent creators will carry legal liability for their agents' actions
“The person who creates the agent probably is gonna, for the foreseeable future, take on a lot of the liability of what that agent does. That agent doesn't deserve any privacy because it's, you know, it can't fully be autonomously operated and it doesn't have a…”
Levie: Autonomous agents require hard-mode governance unlike current user-impersonating tools
“So far we've been in, in easy mode. We've hit the easy button with AI, which is the agent just is you. And when you're in cloud code and you're in cursor and you're in codex, you're just, the agent is you, you're offing into your services. It can do everything…”
Huber: RBAC is dead and AI vendors are ignoring authorization
“Well, I'm just saying, I think, like, RBAC does seem to be dead, right? Like, if you want to say something is dead, probably RBAC is dead. And like the auth story to me seems like incredibly unsolved and unaddressed by like the existing state of like AI vendor…”
Levie: Access governance will decide if enterprise AI agents leak data constantly
“These are like the, some of the most probably, you know, boring problems for 98% of people on, on the internet, but they will be the problems that are the difference between, can you actually have autonomous agents in an enterprise context that are not leaking…”
Levie: Enterprises must adapt their workflows to AI agents, not vice versa
“What's happening is we are changing our work to make the agents effective in that model. The agent didn't really adapt to how we work. We basically adapted to how the agent works. All of the economy has to go through that exact same evolution. The rest of the …”
Levie: Anthropic has forward-deployed engineers embedded at Goldman Sachs
“OpenAI probably is hiring FDEs to go into the enterprise and then Anthropic is embedded at Goldman Sachs.”
Levie: AI agents will force enterprises to organize their data
“Agents will certainly cause us to be much better organized around how we work with our information simply because the severity of the agent pulling the wrong data will be too high. And the productivity gain of that you'll miss out on by not doing this will be …”
Levie: AI agents cannot succeed at search retrieval where smart humans fail
“If a really, really smart human could not do that task in five or 10 minutes for a search retrieval type task, you know, your agent's not gonna be able to do it any better.”
Levie: Language models experience massive degradation after roughly 60,000 tokens
“Today, we have a context engineering problem, which is I got, you know, 200,000 tokens that I can work with, or probably, I don't even know what the latest graph is before, like, massive degradation. Okay, I have 60,000 tokens that I get to work with, where I'…”
Levie: Knowledge work agents require search judgment unlike coding agents
“And that judgment is like a really new thing that the model needs to be able to have is like, when should it give up on a task? Cause, cause you just don't, it's a, can't find the thing. That's the real world of knowledge work problems. And this is the stuff t…”
Huber: Frontier AI models are not actually good at agentic search
“We've like sort of stress tested like frontier models and their ability to search. And they are not actually that good at searching.”
Levie: AI slop in legal contracts creates risks code slop never introduced
“If I have an AI model go generate a contract, and I generate a contract 20 times, and, you know, all 20 times, it's just three percent different. And, like, that, I, that, that kind of slop introduces all new kinds of risk for my organization that the code ver…”
Swyx: The major AI story of 2026 will be knowledge work agents
“I do think like a lot of the last year's, 20, 25 story was the rise of coding agents. And I think 26 stories, definitely knowledge work.”
Huber: Frontier models repeat mistakes if failed actions remain in context
“A few of the insights is, like, everyone, frontier model is not good at search. Humans have this natural explore-exploit trade-off, where we kind of understand, like, when to stop doing something. Also, humans are pretty good at, like, forgetting, actually, li…”
Huber: Model self-pruning of context windows will become standard
“And so, like, I think pruning is also going to be, like, really, it's already becoming a thing, right? But, like, letting models, like, self-prune their context windows.”
Levie: Every enterprise will adopt AI agent evaluation and observability tools
“I've been fully convinced that the whole agent observability and eval space is going to be a massive space. I'm super excited for what brain trust is doing, excited for, you know, Langsmith, all the things. And I think what you're going to, I mean, this is lik…”
Levie: Enterprise software must treat AI agents as a distinct user type
“And then they all are having to understand that now you've got this new customer, which is the agent. And they've been building for two types of customers in the past. They've been building for users and they've been building for like applications. And now you…”
Levie: Succeeding in the AI agent wave is existential for Box
“There's a part of the company that is sort of do or die for the agent wave. And it only happens to be more of my focus simply because it's existential that we get it right.”
Levie: Generating Beautiful PowerPoint Presentations Remains a Hard Problem for AI
“Building a beautiful PowerPoint presentation. It's still a hard problem for these models. Like we still, you know, like these formats are just, we're not built for.”
Levie: Agents Using Box as File Systems Will Outgrow Native Agents
“The thing that I think is, is probably going to be a much bigger The idea over time is any agent on any system, again, using Box as a file system for its work.”
Huber: Most companies operate as apprenticeships with unwritten tacit knowledge
“Most companies are practically apprenticeships. Like every new employee who joins the team, like you spend one to three months, like wrapping them up. All that tested knowledge is not written down.”
Levie: AI will create a huge premium on well-documented companies
“One is I think you're going to see, again, a premium on companies that can document this much. There'll be a huge premium on that because, you know, can you shorten that three month ramp cycle to a two week ramp cycle? That's an instant productivity gain.”
Huber: Graph structures emerge dynamically in AI agents rather than schemas
“I think that the actual graph structure is emergent in the mind of the agent, ah, in the same way it is in the mind of the human. And that's a more powerful graph, because it actually evolved over time.”
Swyx: AI agents should maintain personal wikis for persistent memory
“I think like almost like every agent should have its own wiki that it's updating and that's, Persistent memory. That is a very weak knowledge graph. And you could strengthen it if you want more structure, but you may not need it.”
Levie: A few wrong AI decisions could wipe software companies out
“You can just see how the AI tsunami could wipe you out. If you make just two, three, four or five wrong decisions in this space, like couple wrong architecture decisions, couple wrong AI feature decisions, couple wrong API platform decisions. And you might be …”
Levie: DevRel is surging to help autonomous AI agents discover APIs
“DevRel is hot, like the hottest thing of all time right now. I like, if you could produce a fricking factory of DevRel people, like there's just like unlimited jobs right now on the other end of that. Cause we're gonna, everybody needs their Services and APIs …”
Swyx: Every company must build or buy a media operation
“Every company must either build or buy a media company. Right. Until you, unless you realize that you have to take it that seriously, that you are running a media business in your company. You will never be good at it.”
Levie: Distribution spending will match increased software code efficiency
“The thing that's going to happen on the ledger of software is we're going to produce far more output of code and thus features per dollar. But on the other end of this, we're going to actually end up spending probably just as much on how do you get all of that…”
Levie: Software engineering remains one of the most protected job categories
“I always laugh when, you know, people say, you don't need to be an engineer. Don't do computer science. I actually think like that is like still one of the most protected job categories because things are only getting more technical. Things are only going to g…”
Levie: Most companies lack enough engineers to maintain bespoke internal software
“Most of the economy does not have enough engineers to then maintain all those systems, to update to all those systems, to figure out the relationship between the business problem and what the code needs to do to go and actually manage that.”
Levie: AI agents will generate 10 to 100 times more code
“But no matter what, there's going to be 10 to a hundred times more code. So I think you can be very long engineering right now as just a, you know, purely on the dimension of software is going to become increasingly more important. Once agents are, you know, t…”