Kevin Wang: 1000-layer RL networks can train on single 80GB H100
Kevin Wang · [NeurIPS Best Paper] 1000 Layer Networks for Self-Supervised RL — Kevin Wang et al, Princeton · Dec 31, 2025 · at 24:19
Kevin Wang discusses the computational efficiency of their 1000-layer self-supervised RL network paper presented at NeurIPS.
0:00 / 0:06exact quote · 6.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The nice thing is that all of our experiments, even the thousand layer networks, can be run on one single, 80 gigabyte, each 100 GPU.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Kevin Wang
Kevin Wang: Cross-entropy trajectory classification enables scalable deep reinforcement learning
“I think it's because we're fundamentally shifting the burden of learning from something like, Q-learning or, like, regressing to, like, TD errors, which we know is quite spurious and noisy and biased, to fundamentally, like, a classification problem. We're try…”
Wang: Deep Self-Supervised Method Beats SOTA on Goal-Conditioned RL Significantly
“We do achieve state-of-the-art performance on goal-conditioned RL and Jack's CCRL by a significant amount.”
Kevin Wang: Scaling RL network depth unlocks effective batch size scaling
“We notice that we see that scaling width actually also improves performance, and we also find that actually by scaling depth, we actually unlock the ability to scale along batch size as well.”
Wang: Traditional value-based reinforcement learning fails to scale
“And so what we did is that we know that traditional RL, like let's say like value value-based RL doesn't really scale, right? This is pretty clear from the literature.”
Wang: 64 layers saturate performance in most reinforcement learning tasks
“Within our paper, like, for most environments we are able to, like, saturate, like, get to, like, almost perfect performance within just, you know, we don't even need to get to, like, a thousand layers. Like, maybe just 64 layers, for example, is sufficient.”