Fermat's Library considered tools to flag p-value hacking via API or extension
João Batalha · Fermat's Library Cofounders João Batalha and Luís Batalha · Y Combinator · Jul 6, 2018 · at 16:11
João Batalha, cofounder of paper-annotation tool Fermat's Library, discusses potential product features to help researchers evaluate the statistical rigor of academic papers.
0:00 / 0:27exact quote · 27.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We've thought of doing something with Fermat there, either via some API where you could, like, send us the DOI of a paper, and we would send you, like, some information regarding the p-value or something, or with the Chrome extension where where you'd see that, that information displayed very prominently saying, hey, like, these, this might be, there might be some p-value hacking here, or this is very solid research.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from João Batalha
Batalha: Publishing negative research results could be incredibly useful to scientists
“You might need to change things dramatically in order to get people to start publishing negative results, which are, like, could be incredibly useful. For other researchers.”
João Batalha: Accessible scientific writing enables cross-disciplinary discovery
“You always benefit if you can make it as clear and accessible as possible because you never know, like the audience that is going to be, end up reading your paper. You can, you, of course you can expect that the people in your field are going to read it, but s…”
João Batalha: Major scientific discoveries often spread without accumulating high paper citations
“Sometimes very important well, this is more specific for certain fields, very important concepts or discoveries never really get published in one paper that then gets a ton of citations, or it, that knowledge gets spread, In some other way. And so there are, y…”
João Batalha: Academic peer review requires significant reform to match modern workflows
“The system nowadays, it's, does not seem to be perfect for the way things work nowadays, and we would love to see either via format or some other platform to try to tackle that, and try to do something to make the peer review a better system, or to change it s…”
João Batalha: Paper authors often make poor annotators due to internalized knowledge
“What's interesting is that well, the authors of the paper sometimes, you know, they are not going to know where people are going to struggle understanding the paper, oftentimes.
I remember when I was annotating the Ethereum white paper, right?
Written by Vita…”
João Batalha: arXiv grew linearly, not exponentially, despite huge impact
“If you look at the graph of submissions for Archive, it's completely almost linear. It's, there's no startup exponential, artistic growth. It's like, completely linear, but it's arguably one of the things that has had the most or that has impacted the making o…”