Morris: The next AI paradigm shift will stem from an unused data source
Jack Morris · Information Theory for Language Models: Jack Morris · Jul 2, 2025 · at 1:12:04
Cornell Tech PhD researcher Jack Morris projects how future AI breakthroughs will emerge following previous dataset-driven milestones.
0:00 / 0:14exact quote · 14.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And so whatever the fifth thing is, whether it's Video or embodied AI or some kind of crazy innovation on reasoning models. Whatever comes next will probably be some type of new data source that we're not using yet.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jack Morris
Morris: ChatGPT could likely have been built using RNNs instead of Transformers
“And I think like, we honestly probably could have gotten this with RNNs. I know like the scaling laws paper shows that RNNs have worse curves for scaling, but probably people would have been like, I bet you could have built chat GPT with a very sophisticated R…”
Jack Morris: Fundamental AI science shifted to companies due to academic compute limits
“That's when I think things really started to change in terms of the types of questions you wanted to ask can't always be answered with academic resources. So a lot of the like fundamental kind of like boundary pushing and AI science moved into companies.”
Morris: AI paradigm shifts are driven by novel datasets, not architectures
“I think, like, all of the things that I would consider paradigm shifts in the Kuhnian sense came from a new technique, but trained on new data, and I think the new data is super, super important”
Jack Morris: Most AI research was previously open, but is now closed
“Most stuff was open. Now most stuff is not open.”
Morris: Re-implementing new paradigm shifts quickly is the best AI grad strategy
“Honestly, if I were to give advice to a younger grad student, I think the way to do it would be literally just like sit and wait until the next kind of paradigm shift and then just immediately start working as fast as you can to like re-implement it. Like, I d…”