grokking
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Sep 30, 2025 by Dylan Patel · across every show →
Everything said about grokking, oldest first
Sep 30, 2025 negative
Patel: Scaling model size without new data causes memorization over generalization
“And if you make the model bigger and bigger and bigger without the data changing, you just memorize everything. And actually, it starts to get worse again because it never had the opportunity to generalize because the model was so big and there's so many weigh…”