Nov 9, 2016 · 32m · mad
Artificial Intelligence at Facebook // Antoine Bordes, Facebook [FirstMark's Data Driven]
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
At a DataDrivenNYC event, Antoine Bordes of Facebook AI Research (FAIR) demystifies the state of artificial intelligence by outlining advancements in computer vision, machine reasoning, and natural language processing. He details FAIR's strategic roadmap, open-science philosophy, and how fundamental AI research powers Facebook's long-term ecosystem.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 2.2% of the talking time here. How this is scored →
Matt as informed peer 0.6 Guest teaching 2.2 Guest disagreement 0.3 Matt pushing back 0.1
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 23:35 Reframing ML versus NLP
Bordes gently rejects the premise of the audience question, explaining that ML and NLP are no longer distinct competing paradigms but have fully merged.
Hardest push from Matt ▶ 17:00 Host's check on AI taking overMatt Turck playfully challenges the presentation narrative by asking whether AI is about to take over the world, prompting Bordes to clarify the limits of current models.
Biggest teaching moment ▶ 19:00 Watson vs. FAIR unstructured readingBordes educates the audience on the structural distinction between IBM Watson's heavily curated database approach and FAIR's goal of direct unstructured machine reading.
Matt holds his own ▶ 17:20 Host drilling into team metricsMatt Turck takes control at the end of the presentation, immediately pressing Bordes on concrete lab metrics such as team headcount and organization structure.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Facebook's Scale and the Mission of AI | 0 | 1 | 0 | 0 | Antoine Bordes gives a presentation on Facebook's AI lab mission, explaining how massive daily scale requires automatic translation and computer vision to rank posts. The host is not present during this monologue section. | |
| The Evolution of Neural Networks and Compute Power | 0 | 2 | 0 | 0 | Bordes highlights that modern convolutional neural networks use the same underlying architectures designed by Yann LeCun in 1993, but perform exponentially better due to GPU hardware scaling and massive datasets. The host is silent. | |
| Image Captioning and Its Failure Cases | 0 | 2 | 0 | 0 | Bordes explains why early image captioning models fail when confronted with unusual out-of-distribution scenes, prompting a research pivot toward Visual Question Answering. Monologue segment with no host involvement. | |
| Evaluating Machine Reasoning with the bAbI Benchmark | 0 | 2 | 0 | 0 | Bordes details FAIR's bAbI benchmark dataset of simple synthetic stories designed to test machine reasoning in controlled environments. The host does not speak. | |
| Scaling Reasoning Models from Synthetic Text to Wikipedia | 0 | 2 | 0 | 0 | Bordes describes scaling reading comprehension from synthetic stories to Wikipedia articles and connects it to Facebook's 10-year AI strategy. Monologue segment. | |
| Facebook's 10-Year AI Roadmap and Wrap-Up | 2 | 1 | 0 | 1 | Matt Turck opens the Q&A with a playful check about AI taking over the world, then asks about FAIR's team size and open-source commitments. Bordes responds collaboratively with team counts and lab culture details. | |
| Q&A: IBM Watson vs. FAIR's Unstructured Reading Approach | 1 | 3 | 1 | 0 | An audience member asks how FAIR's reading approach compares to IBM Watson on Jeopardy. Bordes clarifies that Watson relied heavily on structured database pre-processing, whereas FAIR aims for direct reading of raw unstructured text. | |
| Q&A: AI Research Applications in VR/AR and Oculus | 1 | 3 | 1 | 0 | An audience member asks about applying ML models to Oculus VR data. Bordes clarifies that FAIR is a fundamental research lab uncoupled from specific product deliverables or Oculus roadmaps. | |
| Q&A: Machine Learning vs. Natural Language Processing Rules | 0 | 3 | 1 | 0 | An audience member asks whether future text progress will come from ML or rule-based NLP. Bordes reframes the question, noting that ML and NLP have largely merged into a unified discipline. | |
| Q&A: Common Sense in AI and Promising Model Architectures | 1 | 3 | 0 | 0 | Bordes discusses the open challenge of common sense in AI and highlights attention mechanisms like Memory Networks as promising architectures. Turck adds a brief joke about funding a common sense startup. | |
| Q&A: Active Learning and Human-in-the-Loop Systems (Facebook M) | 1 | 2 | 0 | 0 | An audience member asks about active learning and human-in-the-loop systems. Bordes highlights Facebook M as a prime real-world example where human supervision generates rich training data for AI models. |
Statements from this episode (20)
Bordes: Most Facebook users have friends who speak different languages
“Most people are actually friends speaking another language”
Bordes: Pose keypoints and action detection represent edge of AI research
“So the key pose and the action detection is basically at the edge of what we are doing now in the research.”
Most 2016 production computer vision systems lack bounding box capabilities
“This is, right now, I would say that the best systems are doing this in production. Most of them are doing this. And some are not even doing the boxes.”
Bordes: 2016 image recognition AI uses Yann LeCun's 1993 neural architecture
“This is exactly the same architecture of what's being used right now.”
Bordes predicts large AI models will run on mobile devices by 2017
“And next year we will run them on mobile, so.”
Bordes: Image captioning models force unusual situations into common patterns
“If you go to unusual situations, Then, basically, the model is going to try to put this into usual cases as well.”
Visual question answering is harder than captioning because it requires reasoning
“So, what people try to do now is that to move to caption what's called caption generation, which was actually super promising, but actually people realized that actually the machine wasn't that good, to what's called now visual question answering, which is mor…”
Bordes: No AI model can solve bAbI's simple reasoning task
“Basically this one is still unsolved, which is, like, super easy. Actually, no machine can solve this one.”
FAIR shifted from children's book benchmarks to Wikipedia question answering
“And at the end of last year, we also released something that is the same kind of ideas using, like, ah, some short stories, but, ah, that we use from, like, children books, real children books, and now we are moved to a train to answer question, ah, directly f…”
Bordes: FAIR expects single reasoning method to solve all QA cases
“We expect basically the same method to be able to solve all the cases, because we are looking for method that can do reasoning, whether it's a very simple situation on Wikipedia.”
AI cannot yet perform database-style reading and QA on unstructured Wikipedia text
“Nobody can answer on Wikipedia the way that you would answer with a database or anything like this, doing reading.”
Vision, language, reasoning, and planning form Facebook's ten-year AI roadmap
“Understanding vision, understanding language, reasoning, and planning, I didn't even talk about planning, are actually a very key element that fits in the whole Facebook Facebook strategy for the tenures.”
Bordes: Facebook AI Research has about 80 people, mostly researchers
“I mean, we're growing super fast, but I would say we're now 80. 80 people. Like 50 or 60 researchers and the rest of engineers.”
Bordes: Direct machine reading on Wikipedia currently lags IBM Watson
“So of course, in the end, right now, a system just based on Wikipedia is much worse than what can be done by Watson, using all the databases.”
Bordes: Direct machine reading will eventually match IBM Watson's performance
“The goal is that, ah, eventually, by trying to understand the text directly, you can actually at least equal, all the information by Watson is in text, in free text.”
Bordes: Facebook AI Research is not tied to specific product groups
“And actually the lab is not even tied to any application or any product group.”
FAIR prioritizes minimal hand-crafted rules to force machine learning innovation
“Our approach here is basically to try to put as little and crafted features of rule as possible in the system and try to make it learn as much as it can.”
Real-world applications must balance machine learning with rule-based extraction pipelines
“In terms of developing real application, I think you should really try to balance both, because it's true that in the example I showed, If you have a very good process to basically already isolate all the entities and if you want basically to use something lik…”
Bordes: Visual QA models saturate around 60% accuracy versus 95% for humans
“Basically many methods actually saturated at like, I don't know, 55 or 60% accuracy. Ah, the simpler or the most complicated were actually in the same ballpark. Whereas human can actually go up to 95”
Bordes: Facebook M is an AI monitored by human operators who correct errors
“M is actually an AI but supervised by humans. So there we can plug some models and actually there are, but the system is supposed to have very good accuracy and like very high performances. So basically the system is always monitored by humans going to decide …”