Cost Of Inference
topic on 3 shows · 3 statements across 3 episodes
3 statements about Cost Of Inference, every show
AI token costs will asymptotically approach the fundamental cost of power
“I think the answer will increasingly become yes in the future as the cost of inference, the cost of token generation asymptotically approaches the cost of power, which is the only real operational expense input into the Into the cost of intelligence generation…”
Sivulka: AI inference costs dropped 7 orders of magnitude in 4 years
“I think that since Hebbia started, the cost of inference over a fixed number of parameters has decreased by, like, seven orders of magnitude in four years.”
Gerstner: AI inference costs have fallen over 90% in 18 months
“We already know the cost of inference has come down by over 90% over the course of the last 18 months, and our friend Sonny over the weekend when we were in our group chat, you know, said he expects it to come down by another 90% over the course of the next se…”