Packer: Sleep-time compute cannot be brute-forced due to diminishing returns
Charles Packer · Sleep-Time Compute — Letta AI (Charles Packer, Charlie Snell, Kevin Lin) · Apr 21, 2025 · at 14:44
Charles Packer, co-founder of Letta AI, responds to Shawn Wang's question about whether sleep-time compute is merely data pre-processing.
0:00 / 0:43exact quote · 43.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“There's definitely the aspect of, there's diminishing returns. So depending on what you're trying to do, you know, you will kind of reach a limit of how much you can re-represent the context. I think in this case, you know, with these like GSM, AK style questions or like Amy style, there's going to be a limit. Like, you know, at a certain point you will have kind of expanded the context to the point where you've covered like every single foreseeable question you could ask on, on that math problem. But I think that's kind of what's cool about it because I think that also means that it's inherently agentic. So like, you kind of want to be intelligent about how much you actually apply sleep time to compute. This isn't just something you're going to like brute force and, you know, infinitely like get better results on like every single domain.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Charles Packer
Packer: Sleep-time compute during idle downtime is a major missed opportunity
“And practically speaking, you know, machines, they're not like humans, they can be run all the time. And there's a ton of downtime, both in advance of like questions being asked also like After questions have been asked too. So I think beyond just scaling at t…”
Packer: True AI agents run continuously rather than waiting for triggers
“And I think that's another aspect of like what makes something agentic, like not having to have a user send an event to trigger the machine to turn on, just allowing these machines to run all the time.”
Packer: Background sleep-time agent architectures will be standard within two years
“I think those two, yeah, I think similar to memgpt, I think they're definitely like very good reference designs for just what's coming next. I think this sort of thing is, is just gonna be like the norm in like a year or two years.”
Packer: Sleep-time compute re-represents token state into easily queryable formats
“In the test time compute setting, you know, here, the state is tokens and the kind of like sleep time, like indexing process is a re-representation of those tokens into something that is like more easily queryable and more flexible.”
Packer: Sleep-time compute offers Pareto improvements across Claude 3.7 and DeepSeek
“It's like pretty consistent across like both 3.7 deep seek, three mini, which all like the way you actually scale the x-axis here is fundamentally quite different in each case with 3.7 extended thinking mode. The parameter you provide to scale it is different …”