reCAPTCHA required 10 matching user responses to establish ground truth text
Luis von Ahn · a16z Podcast | The Taxonomy of Collective Knowledge · Jan 2, 2019 · at 6:48
Luis von Ahn, founder of reCAPTCHA and Duolingo, discusses the crowdsourcing mechanism reCAPTCHA used to digitize books for Google.
0:00 / 0:15exact quote · 15.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And the way we got the ground truth here is, is simply by having, you know, 10 different people agree with each other. So if you gave the same word to 10 different people and they all agreed that we consider that ground truth that to a very large extent that worked really well.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Luis von Ahn
Luis von Ahn: Paying crowdsourced workers is ineffective due to spam
“I have found that paying people is not so good. Then you really have to spend a lot of effort trying to stop people who are just there to, you know, get your money.”
Luis von Ahn: Many AI systems are just fancy ontologies
“A lot of the things that are quote unquote called AI or artificial intelligence, a lot of times are just fancy ontologies.”
Luis von Ahn: Deep learning requires vast human-entered ground truth data
“The other way in which humans are needed here is to create the ground truth. All of these deep learning or deep AI algorithms need A ton of ground truth in order to get very accurate. It has to be entered by humans.”
von Ahn: In 2017, humans still beat AI at basic image recognition
“Computers are better at playing Go than humans are, but humans are still better at recognizing whether a picture has a cat or not.”
Luis von Ahn: Most of the legal system is extremely inconsistent
“Most of our legal system is extremely inconsistent.”
von Ahn: reCAPTCHA digitized 2 to 3 million books annually
“There are a hundred million books that needed to be digitized. That was the total number of books that has ever been written, you know, before the digital era was one hundred million. At the pace that we were going, we were able to digitize about two to three …”