Token Output

“Memory is, the more memory you put, With flop for a given unit of compute, the more tokens you get out. It's the single most important thing you could do to increase kind of token output per unit of compute.”