Assertion Contradicted
Bordes: 2016 image recognition AI uses Yann LeCun's 1993 neural architecture
“This is exactly the same architecture of what's being used right now.”
Prediction Held up
Bordes: Direct machine reading will eventually match IBM Watson's performance
“The goal is that, ah, eventually, by trying to understand the text directly, you can actually at least equal, all the information by Watson is in text, in free text.”
Insight
Real-world applications must balance machine learning with rule-based extraction pipelines
“In terms of developing real application, I think you should really try to balance both, because it's true that in the example I showed, If you have a very good process to basically already isolate all the entities and if you want basically to use something lik…”
Assertion Not checkable as stated
Most 2016 production computer vision systems lack bounding box capabilities
“This is, right now, I would say that the best systems are doing this in production. Most of them are doing this. And some are not even doing the boxes.”
Prediction Held up
Bordes predicts large AI models will run on mobile devices by 2017
“And next year we will run them on mobile, so.”
Assertion Not checkable as stated
Bordes: No AI model can solve bAbI's simple reasoning task
“Basically this one is still unsolved, which is, like, super easy. Actually, no machine can solve this one.”
Prediction Not checkable as stated
Bordes: FAIR expects single reasoning method to solve all QA cases
“We expect basically the same method to be able to solve all the cases, because we are looking for method that can do reasoning, whether it's a very simple situation on Wikipedia.”
Assertion Not checkable as stated
Bordes: Direct machine reading on Wikipedia currently lags IBM Watson
“So of course, in the end, right now, a system just based on Wikipedia is much worse than what can be done by Watson, using all the databases.”
Disclosure
Bordes: Facebook AI Research is not tied to specific product groups
“And actually the lab is not even tied to any application or any product group.”
Disclosure
FAIR prioritizes minimal hand-crafted rules to force machine learning innovation
“Our approach here is basically to try to put as little and crafted features of rule as possible in the system and try to make it learn as much as it can.”
Assertion Supported
Bordes: Facebook M is an AI monitored by human operators who correct errors
“M is actually an AI but supervised by humans. So there we can plug some models and actually there are, but the system is supposed to have very good accuracy and like very high performances. So basically the system is always monitored by humans going to decide …”
Disclosure
Bordes: Pose keypoints and action detection represent edge of AI research
“So the key pose and the action detection is basically at the edge of what we are doing now in the research.”
Insight
Bordes: Image captioning models force unusual situations into common patterns
“If you go to unusual situations, Then, basically, the model is going to try to put this into usual cases as well.”
Insight
Visual question answering is harder than captioning because it requires reasoning
“So, what people try to do now is that to move to caption what's called caption generation, which was actually super promising, but actually people realized that actually the machine wasn't that good, to what's called now visual question answering, which is mor…”
Disclosure
FAIR shifted from children's book benchmarks to Wikipedia question answering
“And at the end of last year, we also released something that is the same kind of ideas using, like, ah, some short stories, but, ah, that we use from, like, children books, real children books, and now we are moved to a train to answer question, ah, directly f…”
Assertion Supported
AI cannot yet perform database-style reading and QA on unstructured Wikipedia text
“Nobody can answer on Wikipedia the way that you would answer with a database or anything like this, doing reading.”
Disclosure
Vision, language, reasoning, and planning form Facebook's ten-year AI roadmap
“Understanding vision, understanding language, reasoning, and planning, I didn't even talk about planning, are actually a very key element that fits in the whole Facebook Facebook strategy for the tenures.”
Assertion Not checkable as stated
Bordes: Facebook AI Research has about 80 people, mostly researchers
“I mean, we're growing super fast, but I would say we're now 80. 80 people. Like 50 or 60 researchers and the rest of engineers.”
Assertion Partly supported
Bordes: Visual QA models saturate around 60% accuracy versus 95% for humans
“Basically many methods actually saturated at like, I don't know, 55 or 60% accuracy. Ah, the simpler or the most complicated were actually in the same ballpark. Whereas human can actually go up to 95”
Assertion Not checkable as stated
Bordes: Most Facebook users have friends who speak different languages
“Most people are actually friends speaking another language”
