Memory Capacity
topic on 2 shows · 4 statements across 4 episodes
Cheeky Pint 
Invest Like the Best
4 statements about Memory Capacity, every show
Baker: LLM pre-fill is capacity-bound while decode is bandwidth-constrained
“And that is fundamentally a memory capacity bound problem. Decode is the process of generating new tokens, and that is memory bandwidth constraint.”
Patel: Incremental memory fab capacity will not arrive until late 2027 or 2028
“Even if they wanted to build as fast as possible, it doesn't come till 28 early, late 27 at best.”
Pichai: Leading memory makers cannot rapidly expand short-term capacity
“There is no way that the leading memory companies are going to dramatically improve their capacity. So you have those constraints in the short term, but they get more relaxed as you go out.”