Catanzaro: Treat life like gradient descent by iterating rather than over-planning
Bryan Catanzaro · NVIDIA's Artifical Intelligence Moat & Origins — With Bryan Catanzaro · Feb 28, 2024 · at 28:05
NVIDIA VP Bryan Catanzaro draws a philosophical comparison between stochastic gradient descent in neural network training and human decision-making.
0:00 / 0:29exact quote · 29.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I think that you know, you could spend an awful lot of time trying to be very precise about what direction to go to make things better, but often the right thing to do is just make a guess, take a step, And then reevaluate what the best direction is next, and then do that a lot. You know, just be really iterative, really flexible. Don't be too wedded to, you know, the idea that you had about where to go at the beginning of the process. Just let the process kind of guide you as you walk through it.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Bryan Catanzaro
Catanzaro: Humans will primarily interact with AI inside virtual worlds
“And I think that the primary way that people are going to interact with AI is going to be in virtual worlds. Because I think that's going to be the most natural way of interaction in the most useful way.”
Catanzaro: Training data and compute matter more than AI model architecture
“The model is less important than the data. And the compute that goes into training the model. If you have a model that has really excellent compute properties that allows you to scale really well, efficiently to, you know, many thousand of GPUs, the kinds of r…”
Catanzaro: ChatGPT signaled an era where applied AI research dominates academics
“To me, that was a statement that we were entering a new era of AI where applied research starts to dominate, you know, so Chachapiti didn't come out with a fully fledged academic paper that described exactly what they did to make it so awesome. But because the…”
Catanzaro: The greatest risk with AI is failing to adopt it
“I think the scariest thing for me is you know, are we gonna you know, not figure out how to use this technology? Because I think we desperately need it. I think our world desperately needs more intelligence.”