Wang: GPU environments collect hundreds of millions of RL timesteps hourly
Kevin Wang · [NeurIPS Best Paper] 1000 Layer Networks for Self-Supervised RL — Kevin Wang et al, Princeton · Dec 31, 2025 · at 17:43
Kevin Wang (Princeton researcher) discusses how modern GPU simulation environments solve the historical problem of data scarcity in reinforcement learning.
0:00 / 0:05exact quote · 5.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“With these, like, GPU accelerated environments, we can collect hundreds of millions of time steps of data within just a few hours”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Kevin Wang
Kevin Wang: Cross-entropy trajectory classification enables scalable deep reinforcement learning
“I think it's because we're fundamentally shifting the burden of learning from something like, Q-learning or, like, regressing to, like, TD errors, which we know is quite spurious and noisy and biased, to fundamentally, like, a classification problem. We're try…”
Wang: Deep Self-Supervised Method Beats SOTA on Goal-Conditioned RL Significantly
“We do achieve state-of-the-art performance on goal-conditioned RL and Jack's CCRL by a significant amount.”
Kevin Wang: Scaling RL network depth unlocks effective batch size scaling
“We notice that we see that scaling width actually also improves performance, and we also find that actually by scaling depth, we actually unlock the ability to scale along batch size as well.”
Kevin Wang: 1000-layer RL networks can train on single 80GB H100
“The nice thing is that all of our experiments, even the thousand layer networks, can be run on one single, 80 gigabyte, each 100 GPU.”
Wang: Traditional value-based reinforcement learning fails to scale
“And so what we did is that we know that traditional RL, like let's say like value value-based RL doesn't really scale, right? This is pretty clear from the literature.”
Wang: 64 layers saturate performance in most reinforcement learning tasks
“Within our paper, like, for most environments we are able to, like, saturate, like, get to, like, almost perfect performance within just, you know, we don't even need to get to, like, a thousand layers. Like, maybe just 64 layers, for example, is sufficient.”