“And now we're talking about a modality that might end up before the end of the decade being the second or third biggest modality in terms of revenue and potentially also in terms of impact.”
quote is from the automated transcript, cleaned for reading:
filler sounds and stutters are removed, nothing is rephrased. names can be misheard
(the analysis reads context, assessments check outside sources). how →
More from Jakob Uszkoreit
Opinion
Uszkoreit doubts humans will ever achieve full mechanistic understanding of biology
“I don't have very high hopes for humanity to develop that conceptual understanding to the level that we would need it in order to do all the interventions we want to do.”
Jakob UszkoreitAug 24, 2023▶ 16:05No Priors Ep. 29 | With Inceptive CEO Jakob Uszkoreit
Insight
Uszkoreit: Hardware efficiency is the only proven way to advance deep learning
“At the end of the day, in my mind, that's the one and only thing we know really works. If you want to push deep learning forward is to make it faster and more effective and more efficient on a given piece of Hardware.”
Jakob UszkoreitAug 24, 2023▶ 1:29No Priors Ep. 29 | With Inceptive CEO Jakob Uszkoreit
Insight
Uszkoreit: Transformer breakthrough was driven by accelerator hardware fit
“And if you want to look at, say, the biggest differences, for example, between the transformer, as it was described in the attention is all you need paper, and some of its ancestors, like this decomposable attention model, the big difference is just that the t…”
Jakob UszkoreitAug 24, 2023▶ 3:59No Priors Ep. 29 | With Inceptive CEO Jakob Uszkoreit
Opinion
Uszkoreit: GPUs are not at the sweet spot for large-scale deep learning
“I don't think GPUs are at the sweet spot when it comes to large-scale deep learning with respect to exactly those trade-offs, and so it may very well be that if we actually try these combinations, we might actually even quickly find something that's better.”
Jakob UszkoreitAug 24, 2023▶ 5:45No Priors Ep. 29 | With Inceptive CEO Jakob Uszkoreit
Insight
Uszkoreit: Community optimism drove Transformer adoption and success
“The other main contributor, I think, to the success of this architecture was optimism and hope. So suddenly you were in a situation where, for whatever reason, a bunch of things that people tried with this started to work, and then more started to work, and th…”
Jakob UszkoreitAug 24, 2023▶ 7:00No Priors Ep. 29 | With Inceptive CEO Jakob Uszkoreit
Insight
Uszkoreit: Training on synthetic data works by amortizing generation compute
“And ironically, and this comes back to a question that many people ask, I think around, does it make any sense to train on generated data? Because information theory, family information theory, very clearly says, nope, you're not going to get more information …”
Jakob UszkoreitAug 24, 2023▶ 9:24No Priors Ep. 29 | With Inceptive CEO Jakob Uszkoreit
Made with StarZero
Turn any episode into a week of clips.
This entire site, over 100 episodes transcribed, diarized, checked and made playable,
runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the
moments worth sharing, cuts them, captions them, and reframes them for every feed.
We use essential cookies to make the site work. With your permission we
also use analytics cookies (Google Analytics and Mixpanel) to understand
usage and improve StarZero. See our Cookie Policy.