von Ahn: Profiling crowdsourced contributor accuracy enables precision with only 3-4 human checks
Luis von Ahn · a16z Podcast | The Taxonomy of Collective Knowledge · Jan 2, 2019 · at 15:08
Luis von Ahn, founder of reCAPTCHA and Duolingo, explains how platform efficiency is maximized by assigning items to known domain-capable contributors.
0:00 / 0:20exact quote · 20.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“When you're doing things that are very large scale and you don't have very many humans, you may have a thousand humans or 10,000 humans. And if you need to label millions of things. You can't afford to start giving, you know, the same thing to all 10,000 humans. You can only give it to, you know, three or four. And if you can start figuring out who's good at what, you can become a lot more accurate and more efficient.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Luis von Ahn
Luis von Ahn: Paying crowdsourced workers is ineffective due to spam
“I have found that paying people is not so good. Then you really have to spend a lot of effort trying to stop people who are just there to, you know, get your money.”
Luis von Ahn: Many AI systems are just fancy ontologies
“A lot of the things that are quote unquote called AI or artificial intelligence, a lot of times are just fancy ontologies.”
Luis von Ahn: Deep learning requires vast human-entered ground truth data
“The other way in which humans are needed here is to create the ground truth. All of these deep learning or deep AI algorithms need A ton of ground truth in order to get very accurate. It has to be entered by humans.”
von Ahn: In 2017, humans still beat AI at basic image recognition
“Computers are better at playing Go than humans are, but humans are still better at recognizing whether a picture has a cat or not.”
Luis von Ahn: Most of the legal system is extremely inconsistent
“Most of our legal system is extremely inconsistent.”
von Ahn: reCAPTCHA digitized 2 to 3 million books annually
“There are a hundred million books that needed to be digitized. That was the total number of books that has ever been written, you know, before the digital era was one hundred million. At the pace that we were going, we were able to digitize about two to three …”