Parameter Count
topic on 2 shows · 2 statements across 2 episodes
2 statements about Parameter Count, every show
Pope: Model parameter counts will grow much faster than context lengths
“Really tied into this context thing, I think the context size will stay ballpark the same way it is, maybe a few times larger. But the parameter count will go up. Like, parameter count should grow much, much faster than context length, actually, just because o…”
Gurley: LLM scaling on parameters and context windows could hit ceilings
“I look at the LLM models. I mean, they've scaled up on parameter count and the width of the attention window. There easily could be limitations to that. Like if you just think about how optimization models work, they could run out. Like it's not infinite scali…”