PaleoDeepDive matches or exceeds human data extraction accuracy across all tasks
Chris Ré · Chris Re · Jul 15, 2017 · at 13:41
Stanford Professor Chris Ré compares the data extraction accuracy of PaleoDeepDive to human volunteers across various paleontological data predicates.
0:00 / 0:07exact quote · 7.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“In some predicates, it's basically a wash. There's information that humans and machines are equally good at. But we're never any worse, and in some cases, we're substantially better.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Chris Ré
Probabilistic inference can reduce data pipeline engineering effort by orders of magnitude
“In contrast, we can build a pipeline first, which may not be very high quality, and then use probabilistic inference as a way to say, where should we spend our effort next? And the reduction in effort can sometimes be orders of magnitude, as we've seen in some…”
Classical models of sequential evaluation can be completely thrown away
“That is, why it's a new trade-off space and classical trade-off space don't sort of suffice, and to do that, I'm going to show you one result which just illustrates the classical models of sequential evaluation you can completely throw away.”
Relaxing hardware memory consistency yields up to 1000x speedups over competitors
“We've basically done a couple of parlor tricks, and those parlor tricks have allowed us to get, you know, 10, a hundred, a thousand times faster than competitor systems, because we're actually taking advantage of what the hardware is giving us.”
Automated dark data pipelines can outperform human data extraction quality
“There are a number of systems that have shown that it's actually possible to build these ETL pipelines, these pipelines that extract, transform, and load information with higher quality than humans in some very simple settings.”
Traditional data pipelines over-extract and over-clean due to unmeasurable impact
“So in contrast, if you think about the way that people build these extraction and integration and cleaning systems, they tend to over-extract, over-integrate, and over-clean Because they have no idea if their cleaning or extraction is actually gonna prove the …”
Modern scientific literature is universally accessible but impossible for humans to read
“For even the narrowest of scientific questions, They couldn't possibly read all the information that was relevant to them, or even a significant fraction of it. So this is why I say that scientific knowledge is accessible in a way like never before, but it's n…”