Entropy
topic on 1 show · 1 statements across 1 episodes
1 statements about Entropy, every show
Karpathy: Synthetic data risks silent distribution collapse without injected entropy
“When you're doing synthetic data generation, this is a problem, because you actually really want that entropy. You want the diversity and richness in your data set. Otherwise, you're getting collapsed data sets, and you can't see it when you look at any indivi…”