Inference Compute
topic on 4 shows · 6 statements across 6 episodes
BG2 Pod 
Latent Space 
No Priors 
Big Technology
6 statements about Inference Compute, every show
Allaire: AI inference compute could create productive proof-of-work crypto
“The idea of essentially, like, inference compute as GPU inference compute as proof of work. And so the work itself is the inference. And that as the underlying basis for proof of work cryptocurrency is pretty interesting. And would be, you know, potentially so…”
Merrill: AI agents could ace Terminal-Bench with $1,000 compute per task
“Like, I think there's probably a way in which you spend a thousand dollars on tokens for each terminal bench task. And like get them all right just by scaling inference time compute.”
Huang: Inference Compute Demand Will Surge One Billion Times
“It's about to go up by a billion times.”
Lambert: Hybrid reasoners may be phased out except for niche uses
“I think in plenty of ways, like hybrid reasoners might just be aged out except for niche applications because quality is so much more important than having a hundred X less inference tokens. It's like you just pay for it and compute and that'll get better.”
Levie: Multi-agent inference architectures will become an industry standard
“They have a model called Grok for Heavy that has multiple agents go off and execute basically the same task and then go, they go and review their answer for, Which answer they think is, these agents think are the best result. And so this is a great example of …”
Gil: Future AI reasoning will shift to inference-time compute and retraining
“And I think that's also a little bit under discussed in terms of probably a lot of what's going to happen in the future, particularly we get into agents and reasoning is stuff that's happening at that point of inference. And then it's used to sort of feedback …”