Liang: Language models should use calculators instead of computing internally
Dr. Percy Liang · No Priors Ep. 7 | With Stanford Professor Dr. Percy Liang · Apr 25, 2023 · at 16:09
Stanford computer science professor Dr. Percy Liang discusses the integration of external tool use versus native computation inside transformer foundation models.
0:00 / 0:16exact quote · 16.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“There are cases where you want to just map natural language into say people call it tool use. Like you ask some question that reverse calculation, you should just use a calculator rather than trying to sort of quote unquote do it in the transformers head.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Dr. Percy Liang
Liang: The AI industry is retreating from its open culture
“And what we're seeing now is sort of a retreat of that open culture where models are now being only accessible via APIs. We don't really know all the secret sauce that's going behind them, and there's sort of limited access.”
Liang: Big tech scale forced AI academia to focus on understanding models
“And now today I think it's the dynamic is, is quite different because it's no longer academia's job isn't just to get things to work because you can do that in other ways. There's a lot of resources going into big tech companies where there's if you have data …”
Liang: AI benchmarks should target superhuman reliability over human mimicry
“I think we're getting to a point where along many axes, it's a superhuman or should be superhuman. And I think we should maybe define more of an objective measure of like what we actually want. We want something that's very reliable, is grounded. You know, I o…”
Liang: LLMs are not just memorizing because novel concept fusion requires invention
“You know, people say that sometimes all language models just memorize because they're so big and train on clearly a lot of texts, but these examples, I think really indicate that there's no way That these language models are just memorizing because this text j…”
Liang: Next-token prediction forces language models to build world models
“If you think about predicting the next word, It's, it seems very simple, but you have to really internalize a lot of what is going on in this context. What are the previous words? What's the syntax? What's who's saying them? And all of that information and con…”
Liang: Foundation Models Cause the Concept of an AI Task to Dissolve
“And this was a paradigm shift, in my opinion, because it changed the way that we conceptualize Machine learning and NLP systems from these bespoke systems where you're, it's trained to do question answering, to train to do this, to just a general substrate whe…”