Jun 10, 2020 · 1h 0m · mad
Fireside Chat: Jerome Pesenti (Head of AI, Facebook) with Matt Turck (Partner, FirstMark)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this fireside chat hosted by Matt Turck of FirstMark, Facebook Head of AI Jerome Pesenti details Facebook's AI organizational structure, multimodal content moderation, open-source strategy with PyTorch, hardware compute constraints, and practical AI safety.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 19.3% of the talking time here. How this is scored →
Matt as informed peer 3.4 Guest teaching 3.3 Guest disagreement 1.3 Matt pushing back 0.8
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 46:13 Disagreement with Michael Jordan's AI terminology critique
Jerome explicitly rejects UC Berkeley Professor Michael Jordan's assertion that current technology is merely intelligent automation, defending the AI label by citing breakthroughs in games, image recognition, and Blenderbot.
Hardest push from Matt ▶ 34:04 Host pushes to quantify training compute costs in dollar figuresMatt refuses to accept vague descriptions of compute scale, directly prompting Jerome to confirm that single training runs for advanced AI models now reach millions of dollars.
Biggest teaching moment ▶ 36:30 Deconstructing the concept of AGI and human intelligenceJerome reframes Matt's question on AGI by dismantling the premise itself, explaining that human intelligence is highly specialized to survival rather than truly general, and arguing that AGI discourse distracts from present-day algorithmic safety.
Matt holds his own ▶ 30:12 Host demonstrates technical understanding of compute scale limitsMatt cites Jerome's offline technical presentations to articulate the economic and hardware bottlenecks facing brute-force deep learning models.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Organizational Structure of Facebook AI | 3 | 2 | 1 | 1 | Matt demonstrates knowledge of Facebook AI's organizational scale by asking if FAIR still operates around 300 researchers across eight offices. Jerome gently reframes without disclosing precise headcount, confirming the ballpark while outlining the core pillars. | |
| Facebook as a Deep Learning Driven Company | 4 | 3 | 1 | 1 | Matt prompts a discussion on multimodal moderation by describing image-text contradictions in meme content. Jerome elaborates using slides on transformer architectures like XLM-R and multimodal fusion networks. | |
| Moderating Misinformation & COVID-19 Information Integrity | 4 | 3 | 1 | 1 | Matt cites specific initiatives such as SimSearchNet and the DrivenData deepfake contest. Jerome explains fingerprinting, similarity embeddings, and ensemble methods used to track misinformation. | |
| Product Innovations: COVID Response, Facebook Rooms & AR Effects | 3 | 3 | 0 | 0 | Matt asks whether COVID-19 accelerated specific product launches, prompting Jerome to detail Facebook Rooms, AR backgrounds, and Carnegie Mellon symptom tracking initiatives. | |
| Facebook Shops & Universal Product Understanding | 2 | 2 | 0 | 1 | Matt asks if Universal Product Understanding features are currently live or under construction. Jerome clarifies that background AR is live while shopping features roll out over coming months. | |
| Open Source Strategy, PyTorch & Reproducibility in AI | 4 | 3 | 1 | 0 | Matt highlights Facebook's open-source collaboration with AWS on TorchServe. Jerome explains Facebook's strategic rationale for open sourcing models to ensure reproducibility and community progress. | |
| Introducing Blenderbot: Open Source Conversational AI | 2 | 4 | 0 | 0 | Jerome educates the host on how Blenderbot's 9B parameter open-domain approach differs fundamentally from intent-recognition scripted dialogue bots like Alexa or Google Home. | |
| AI Compute Scaling, Hardware Constraints & Efficiency | 5 | 4 | 1 | 2 | Matt demonstrates deep background preparation by citing Jerome's prior talks on compute consumption bottlenecks. Matt then pushes Jerome to quantify training costs in millions of dollars. | |
| AGI Debunked & Real-World AI Safety | 4 | 5 | 4 | 1 | Matt introduces Jerome's Twitter debate with Elon Musk on AGI. Jerome strongly reframes the debate, rejecting the term AGI and explaining that human intelligence itself is highly specialized rather than general. | |
| Advice for AI Startups & High-Impact Applications | 3 | 3 | 1 | 0 | Matt references Jerome's entrepreneurial history at Benevolent to frame startup advice. Jerome outlines high-leverage domains in scientific discovery and creative tools. | |
| Audience Q&A: Data Literacy, Bias, Datasets & System Drift | 3 | 4 | 3 | 1 | In response to an audience question citing UC Berkeley Professor Michael Jordan's critique of AI terminology, Jerome explicitly disagrees, asserting that modern vision and language breakthroughs merit the term AI. | |
| Audience Q&A: Techniques, Democratization, ONNX & Governance | 4 | 4 | 2 | 2 | Matt presses on political ad personalization policy, which Jerome defers. Jerome also pushes back against low-code hype, arguing programming skills remain necessary to build AI systems. |
Statements from this episode (24)
Pesenti: Facebook has more ML engineers outside core AI team than inside
“And actually there are more ML engineers outside my team than within.”
Pesenti: FAIR lab had about 300 researchers across eight offices in 2020
“We don't share, but you're yes. Bullpark is good. Yeah.”
Pesenti: Facebook employs 30,000 content moderators
“So we have actually 30,000 moderators that try to really understand if the content that's published on a platform satisfy our policy, but 30,000, given the number, the amount of content, and you're talking about billions of pieces of content every day is not e…”
Pesenti: Facebook trains single XLM-R models across 100 languages simultaneously
“We try to learn a hundred languages at the same time in a single model using transformer architecture”
Pesenti: Facebook uses embedding algorithms to automatically match misleading content
“And then when they fly content that should be at least, you know, shown as a misleading, then we have this very advanced similarity algorithm that kind of like look at an embedding of the of the content itself. Again, it can be the image itself or a multimodal…”
Pesenti: Deepfake detection will never be a solved problem
“It's never going to be a soft problem, right? So it's always going to be an arm race. I'm confident that we can actually identify part of it. I think we can act on it. We can label it in the case where it matters to users, but I'm also confident that it's goin…”
Pesenti: Facebook gained a year of usage growth in one month
“Basically the growth that happened in, just a matter of a month was what we predicted would happen over more than a year”
Pesenti: Facebook is not an AI vendor and has no cloud offering
“So for us, actually, we're not selling AI. We're not a vendor. We don't have a cloud offering.”
Pesenti: AI will not enter another winter anytime soon
“In my opinion, you know, we're not going to go to any AI winter anytime soon.”
Pesenti in 2020: 9B parameter Blenderbot is the largest chat model
“It's the largest chat model out there. It's nine billion parameters.”
Pesenti: Alexa and Google Home rely on limited, scripted dialogues
“When you interact with something like Alexa or Google Home today, right, this system basically do intent recognition and then they have a very scripted dialogue after this, right? So usually You recognize one of the steps, and it's very limited because you can…”
Pesenti in 2020: Large neural chatbots are not ready for consumer products
“It's not ready for prime consumption, let's be clear. You know, like these bots have lots of interesting side effects. You know, you cannot really control what they're gonna say. But they are quite entertaining, quite engaging, and a lot more human-like than y…”
Pesenti in 2020: 10x annual AI compute scaling is economically unsustainable
“So we're still pushing hard, but it's not going to be 10 X per year. It's just not sustainable.”
Pesenti: Inference remains Facebook's largest machine learning cost
“Actually, To be clear, the most costly thing we do in ML is still inference cost, because when you put a piece of content within Facebook, it's running hundreds of different ML based algorithm, and they all run on machine parallel, and it's using a huge number…”
Pesenti in 2020: Multi-million-dollar AI training runs are unsustainable for Facebook
“Where it becomes millions is when you do training runs. So some of the training runs in the most advanced system that it comes from our company or other companies out there are starting to be extremely expensive. Yeah. Like you can look at one run in the scale…”
Pesenti: AI matching human intelligence is at least 20-30 years away
“And I don't believe anybody has a good view as to when will match human intelligence, but it's not going to happen in the next decade, not going to happen in the next two or three decades. It's going to take much longer.”
Pesenti: General intelligence does not exist, as human intelligence is domain-specific
“Human intelligence is no, it's not general. It's actually very, very specific to our world. It's very biased. It's really customized to survival human in, on our planet. So the concept of general intelligence, I mean, yes, humans tend to have a more general in…”
Pesenti: AI-assisted scientific discovery will break through in coming years
“I do think that AI-assisted science is, has a huge future ahead of us, you know, that it's chemistry, biology, physics leveraging AI system to make better discovery is something that's gonna really break through in the coming years.”
Pesenti: Facebook AI transformer models matched or beat Mathematica at math
“My team also applied transformer models to mathematics, you know, doing, like, things like partial derivation equation or integration and showing that, hey, it could work as well better than Mathematica or some, you know, hundred page long algorithm.”
Pesenti: Facebook has as many data scientists as product managers
“So actually, you know, to get an idea, there are as many data scientists in Facebook as they are product managers, right?”
Pesenti: Do not train production AI chatbots on Reddit data
“So I would not advise to train a production bot on Reddit.”
Pesenti: Self-supervised learning will supersede other AI techniques across fields
“Self-supervised learning is really showing promises everywhere, you know, so it's obviously that's the technique behind large language models, but our, you know, my team came up with new papers around vision. So we believe actually that these techniques will s…”
Pesenti in 2020: AI will not become accessible to non-programmers anytime soon
“I don't buy that AI will be accessible to people who don't have a programming background anytime soon, right? So, but it will be part of, you know, machine learning will be part of the tool set of Any programmers, but it would still be, you know, for kind of m…”
Pesenti: Facebook is going end-to-end all-in on PyTorch
“We're definitely going all in as PyTorch, you know, end to end. So I think initially when we launched the Onyx strategy, it was more like a multi-framework world. And we had actually two framework internally between PyTorch and Cafe Two, but we're still suppor…”