The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 38 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Hilary Mason: Startups must understand how their machine learning models work
“Won't speak for the large companies or people who are working on very specific problems, but when you're generally building systems that in a startup environment, it's really important to understand why your system is doing the thing it's doing, or else it's g…”
Mason: Deep learning completely removes the need for domain feature engineering
“In deep learning, you're not doing feature engineering anymore. Nobody cares about what you know about the domain.”
Mason: Sentiment analysis remains fundamentally unsolved due to lacking ground truth
“I think there are a couple of things that everyone thinks are solved problems that are absolutely not, and so something like sentiment analysis is something where we sort of take for granted that you can just plug into an API and get a number back, but this is…”
Hilary Mason: Separating R&D from product rarely works for tech companies
“I have seen companies that successfully separate, ah, research and development as long as everyone still eats lunch together but generally that doesn't seem to be a great approach in that you eventually end up with a research department off in the corner publi…”
Hilary Mason: Data science sits at the intersection of math, engineering, and hacking
“I tend to think that data science exists in the middle of all of these things. So math, statistics, computer science, and the ability to write algorithms, engineering, and then hacking”
Hilary Mason: Bitly data shows zero overlap between NYT and Fox News readers
“Nobody who reads the New York Times reads Fox News.”
Hilary Mason: Hadoop is entirely impractical for real-time products
“All it is is a structure for running queries in parallel against data that you store in a redundant file system, and so it is entirely impractical for doing a real-time product.”
Mason: Machine learning startups struggle with innovation due to lack of data
“There are challenges for startups, because you don't have data.”
Mason: Building generic ML products is harder than solving single enterprise problems
“When somebody, when a vendor or a startup is going to build a product that solves your problem, They must solve a generic formulation of the problem. They have to solve everybody's version of your same problem. When you want to solve your problem, you just nee…”
Mason: Most tech breakthroughs apply existing research to entirely new domains
“It most often is something where there's been research in one domain that is applicable to another domain. So it's not necessarily something entirely new, but rather somebody Discovering something that may have existed for a while, but realizing that it's appl…”
Mason's 2016 prediction: Deep learning compute will be trivially affordable by 2018
“These days you can afford it and I expect in a year or two it'll be trivial to afford it.”
Mason: Open source is rapidly commoditizing machine learning capabilities
“We're seeing this in machine learning primarily in the open source world in an ongoing basis, almost something new every day.”
Mason: Wikipedia data secretly underpins almost every open machine learning API
“If you go back far enough in pretty much any sort of open API in machine learning, you can find Wikipedia in there somewhere.”
Mason: Natural language generation helps us comprehend data, not replace reporters
“The real impact here is not in replacing reporters but in helping people understand complex data.”
Mason: Real estate valuation is fundamentally an inference problem
“Real estate is a fantastic inference problem because the only way to know the value of a property is to sell it, but obviously we don't sell every property every day”
Mason: Publications reflect past research, while informal chats reveal future ideas
“And so I really believe if you want to know what someone did a year ago, you read their publication, and if you want to know what they're doing now, you watch them give a talk like this, and if you want to know the sort of wacky ideas they're thinking about, y…”
Mason: Complex corporate data problems are either technical or organizational
“Mostly I just sort of talk to companies that have interesting or hard data problems, and those data problems tend to take two forms. They tend to be either highly technical You know, hours of whiteboarding infrastructure and coming up with ideas and that's rea…”
Mason: A company is infrastructure deployed to scale beyond oneself
“A company is a piece of infrastructure that you can deploy when you need to scale something beyond yourself.”
Hilary Mason: Data science deserves its own title combining math, code, and communication
“Data scientists as a job does deserve its own job title because these three things in one professional is new.”
Mason: The vast majority of math in data science is quite simple
“So at least in my practice of data science, the vast majority of the math we did was quite simple.”
Mason: Data engineering designs systems dependent on flowing data
“Data infrastructure. Or data engineering, which is the engineering of systems where the design of the system itself is dependent on the nature of the data that flows through it”
Mason: Forcing teams to admit informal error metrics motivates analytical rigor
“And if there are no error metrics that we are planning to use initially you know, we might be working on something like a recommendation algorithm. And the error metric for the first version is, it looks good to me. You at least have to admit that in public, i…”
Mason: Data projects must prove immediate relevance to business goals
“Assuming we can solve this perfectly, what is the first thing we'll do with it that makes sure it has immediate relevance to the product, or the business, or the system you're building? Because there are a lot of super cool ideas that do not tie directly to th…”
Mason: Asking how an evil dictator would use product sparks creative ideas
“So if you ask this question, like, if we put this thing in the hands of an evil dictator, ah, what would they do with it? Ah, it's a way to get people to sort of brainstorm and come up with crazy ideas for it, some of which might actually be good.”
Mason: Relying on peers for data hurts efficiency; build self-serve dashboards
“Whenever somebody has to ask someone else for a piece of information to get their job done, it really makes everybody much less effective. So if you see people asking for the same kinds of information, there should be dashboards or a way that they can access i…”
Hilary Mason predicts commoditization of tools for processing Big Data
“I think we'll see a change in the vocabulary and a commoditization of the tools to make that happen.”
Mason: Bitly data shows isolated consumption for adult, religion, and chemistry content
“People who look at adult content only look at adult content. People who look at religion only look at religion, and people who look at chemistry really only look at chemistry.”
How the Associated Press used algorithmic systems to automate financial reporting
“They're working with a company called Automated Insights to write a bunch of finance stories. They actually have someone called Automation Editor whose job it is is to manage these algorithmic systems generating content.”
NYC apartments described as 'cozy' average 400 square feet smaller than normal
“If it says cozy, it is 400 square feet smaller than the average for that zip code.”
Mason: The ImageNet dataset completely lacked training data for NYC subways
“There were, was no training data in the ImageNet data set of the New York City subway system.”
Hilary Mason: Data analysts should investigate areas where intuition fails
“Generally, look for places where intuition fails. That is, ah, your company, your product, your data, you should have a good sense of what you would expect. And yet, when you see something in it that doesn't meet your expectations, ah, that's probably an indic…”
Mason balances data teams with one predictable and one speculative project
“And in terms of the day-to-day management, I'd like to make sure everyone who I work with has both, you know, one sort of longer-term, more well-understood problem, and then one problem they can work on when they're really excited, ah, that's a little bit, as …”
Hilary Mason: Bitly processes tens of millions of unique URLs daily
“Tens of millions of unique URLs per day, hundreds of millions of clicks on those URLs from every imaginable social network.”
Mason: Link half-life is 2.8 hours on Twitter versus 3.1 on Facebook
“We found that for Twitter, it's short, it's 2.8 hours. For Facebook, it's a little longer, about 3.1 hours.”
Mason: 3% of all 2011 Bitly clicks went to top 100 celebrity pages
“Three percent of all clicks in 2011 went to pages about the top hundred celebrities.”
Hilary Mason: Bitly's real-time stack relies on C, Python, and Redis
“A lot of our stuff is homegrown, unfortunately. A lot of it's written in C. The rest is in Python. We use Redis quite a lot as a data store.”