Mann: Output verification will be the primary blocker for enterprise LLM deployments
Gideon Mann · A Conversation on The State of AI | Melanie Kambadur, Meta & Gideon Mann, Bloomberg · Feb 2, 2023 · at 14:01
Gideon Mann, Head of Machine Learning at Bloomberg, argues that evaluating whether LLM outputs are accurate and effective remains a major bottleneck for AI adoption.
0:00 / 0:13exact quote · 13.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I think that's gonna be one of these evergreen problems, ah, for LLMs, and we're gonna, you know, keep trying to chew on that, ah, for a while, but that's gonna be the big blocker of a lot of the further deployments.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Gideon Mann
Mann: GitHub Copilot saves developers about 40 percent time on new code
“They've done some studies, and it's, they, studies suggest that it saves about 40% for new code for developers to write new code.”
Mann: Every software application user interface will eventually integrate an LLM
“You can imagine that every point of interface with software application, there's gonna be a point to have a large language Model in that point of interface, and I think all of those will be interesting and useful”
Mann: Human intelligence is overrated, making the path to AGI more plausible
“I kind of think human intelligence is a little bit overrated. So I think I'm a little more bullish on AGI. Someone wrote, you know the large, I think it was Ilya Sutskever, you know, maybe the largest models are slightly conscious. I'm not sure I disagree. So …”
Mann: AI breakthroughs are an overnight success seven years in the making
“It, for me it feels like an overnight success, seven years in the making.”
Mann: Open source culture in academic machine learning accelerated AI innovation
“The academic machine learning community happened to be one that shared quite a bit. And so companies like Hugging Face and, you know, Google's Colab, you know, enabled sharing of code sharing of technology, and this just really increased the pace of innovation”