Feb 2, 2023 · 21m · mad
A Conversation on The State of AI | Melanie Kambadur, Meta & Gideon Mann, Bloomberg
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Host Matt Turck moderates a Data Driven NYC panel featuring Melanie Kambadur of Meta and Gideon Mann of Bloomberg as they discuss the key technological drivers behind recent AI breakthroughs, current enterprise applications, critical deployment challenges like hallucination and compute costs, and the future path toward Artificial General Intelligence.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 13.8% of the talking time here. How this is scored →
Matt as informed peer 2.4 Guest teaching 4.4 Guest disagreement 0.1 Matt pushing back 0.9
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 20:02 Gideon challenges host's framing of human intelligence
Gideon playfully counters the host's cautious framing on AGI difficulty by claiming human intelligence is overrated and suggesting large models may already exhibit slight consciousness.
Hardest push from Matt ▶ 14:15 Matt challenges the timeline for fixing LLM flawsMatt challenges the optimistic narrative by asking if solving the final 10% of problems like hallucinations will be extraordinarily painful and protracted compared to initial gains.
Biggest teaching moment ▶ 0:29 Gideon details the seven-year stack behind AI breakthroughsGideon reframes Matt's question about recent sudden breakthroughs by delivering a detailed breakdown of hardware, software libraries, transformer architectures, and open-source culture.
Matt holds his own ▶ 15:20 Matt clarifies technical terminology for listenersMatt demonstrates his domain fluency by interrupting to concisely define inference versus training for the audience before Melanie continues her breakdown of breakdown of generation costs.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Key Drivers Behind Recent AI Breakthroughs | 1 | 5 | 0 | 0 | Matt opens with a broad high-level question asking what is driving the recent AI boom. Gideon provides an extensive technical history covering GPUs, PyTorch automatic differentiation, transformer architectures, open-source sharing, and parameter scaling from GPT-2 to GPT-3. | |
| The Evolution and Scalability of Multimodal AI | 2 | 5 | 0 | 0 | Matt asks a guided question about why multimodal AI is suddenly advancing across text, images, and video. Melanie explains the combination of data scale, multimodal dataset quality, and specific breakthroughs like voice replication and short-prompt video generation. | |
| Exciting AI Applications and Natural Language Interfaces | 2 | 4 | 0 | 0 | Matt asks the guests to name interesting applications versus overblown hype in the field. Melanie and Gideon detail real-world production deployments, including Character AI, 3D metaverse generation, and Bloomberg's internal search transition from regex/parsers to LLMs. | |
| Major Technical and Business Gaps in Deploying LLMs | 3 | 5 | 0 | 1 | Matt asks where the gaps remain between hype and production readiness. Melanie outlines compute constraints, hallucinations, safety issues, and data licensing, while Matt contributes a sharp, humorous analogy comparing model hallucinations to venture capitalists. | |
| Case Study on GitHub Copilot and Output Verification | 0 | 4 | 0 | 0 | The host remains silent in this segment while Gideon interacts directly with the audience regarding GitHub Copilot adoption. Gideon explains that developer output verification is the key enabling factor for Copilot and a major hurdle for broader LLM deployment. | |
| Overcoming Safety Risks, Hallucinations, and Inference Costs | 5 | 4 | 0 | 3 | Matt pushes the guests on whether AI progress faces an 80/20 difficulty curve and clearly defines inference versus training for the audience. He also questions whether current AI acceleration is exponential or reaching a temporary plateau, prompting technical analyses from both guests. | |
| Self-Training AI Models and Expert Human Feedback | 4 | 4 | 1 | 2 | Matt frames the closing debate around AGI by contrasting brute-force scaling against explicit reasoning architectures. Gideon playfully pushes back on the premise by arguing human intelligence is overrated, while Melanie highlights expert RLHF and synthetic data self-training. |
Statements from this episode (17)
Mann: AI breakthroughs are an overnight success seven years in the making
“It, for me it feels like an overnight success, seven years in the making.”
Mann: Open source culture in academic machine learning accelerated AI innovation
“The academic machine learning community happened to be one that shared quite a bit. And so companies like Hugging Face and, you know, Google's Colab, you know, enabled sharing of code sharing of technology, and this just really increased the pace of innovation”
Mann: GPT-3 was vastly better than decades of prior NLP systems
“Two years ago, I remember being shocked at GPT-III when it was released, because it was just so, I'm an old NLP guy, and it was so vastly better than anything that, that we could build for decades.”
Kambadur: AI voice models replicate human speech from three seconds of audio
“So now we have voice models that can listen to Three seconds, I think, that we had recently. This new volley of your voice, and then replicate your voice. We have models that can get, like, a very short sentence and generate, like, minutes-long video that desc…”
Kambadur: Generative 3D AI tools will vastly accelerate building the metaverse
“That's kind of like how the metaverse is gonna happen. We're gonna be able to like walk around and generated three D worlds in VR, and they're all gonna be built Much more quickly than maybe we expected a few years ago.”
Mann: LLMs will outperform decades of prior enterprise search techniques
“When I look at the, you know, all of the LLM technology, large language model technology, it seems like it will get closer than any of the things that, that, you know, the field has been doing for the past few decades.”
Kambadur: Stack Overflow banned ChatGPT for generating plausible but incorrect answers
“ChatGPT got banned from stack overflow because it sounded convincing, but wasn't quite right all the time.”
Mann: GitHub Copilot is likely the most widely deployed LLM application
“So far the most widely deployed application of the large language models is probably Copilot.”
Mann: GitHub Copilot saves developers about 40 percent time on new code
“They've done some studies, and it's, they, studies suggest that it saves about 40% for new code for developers to write new code.”
Mann: Output verification will be the primary blocker for enterprise LLM deployments
“I think that's gonna be one of these evergreen problems, ah, for LLMs, and we're gonna, you know, keep trying to chew on that, ah, for a while, but that's gonna be the big blocker of a lot of the further deployments.”
Kambadur: Triggering unsafe OpenAI outputs requires clever adversarial prompt engineering
“I've seen really huge advances from open AI's models in their responsible AI and their safety. Like now it takes a lot of clever prompt engineering is what we call, you know, using different ways of talking to the model to get it to reveal interesting stuff. I…”
Kambadur: AI inference costs tens of cents for text, dollars for video
“Every time the user wants to call the model, it might cost, you know, tens of cents for a text model or dollars for a video model.”
Mann: Unoptimized model pipelines mean AI development is still in early stages
“I think we're still early. I think On, on, on, for two reasons. I think one is, ah, the, you know, to Melanie's point, the amount of tuning and optimization across the entire pipeline is, is really premature.”
Mann: Every software application user interface will eventually integrate an LLM
“You can imagine that every point of interface with software application, there's gonna be a point to have a large language Model in that point of interface, and I think all of those will be interesting and useful”
Kambadur: OpenAI sought top 10th percentile doctors and lawyers as data annotators
“Their model itself has gotten so good that they now need, like, extreme experts to be able to train the model, so we saw an ad for their annotators, which are the people who, like, help give feedback to the model, and they asked for, like, top 10 percentile ch…”
Kambadur: Synthetic data generation is a highly popular use case for LLMs
“A very popular use of these LLMs is to generate synthetic data to train other models, because they're really good at generating sort of long tail data.”
Mann: Human intelligence is overrated, making the path to AGI more plausible
“I kind of think human intelligence is a little bit overrated. So I think I'm a little more bullish on AGI. Someone wrote, you know the large, I think it was Ilya Sutskever, you know, maybe the largest models are slightly conscious. I'm not sure I disagree. So …”