Dec 8, 2016 · 24m · mad
A Process for Discovery // Hilary Mason, Fast Forward Labs [FirstMark's Data Driven]
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
At Data Driven NYC, Hilary Mason, Founder and CEO of Fast Forward Labs, delivers a comprehensive talk on bridging academic machine learning research with commercial enterprise applications. She details her firm's structured methodology for identifying emerging technology trends and demonstrates functional prototypes across natural language generation, computer vision, and probabilistic programming.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 2.2% of the talking time here. How this is scored →
Matt as informed peer 0.0 Guest teaching 3.9 Guest disagreement 1.0 Matt pushing back 0.0
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 21:40 Challenging common wisdom on sentiment analysis
Hilary directly refutes industry consensus, stating that sentiment analysis is widely assumed to be a solved problem when it actually lacks basic ground truth and reliable tool consensus.
Hardest push from Matt ▶ 21:25 Host probing for unready technologiesMatt Turck pushes past the presentation's success stories to explicitly ask Hilary which emerging technologies are hyped as ready but actually fail to deliver in practice.
Biggest teaching moment ▶ 15:55 Explaining neural network dataset blind spotsHilary uses her personal experience with Instagram photo classification to demonstrate how a lack of subway images in ImageNet forced a deep learning model to misidentify subway stations as prisons.
Matt holds his own ▶ 21:20 Host counter-probing technology readinessMatt Turck demonstrates keen industry knowledge by challenging the guest to reveal which technologies only appear ready for prime time versus those that truly are.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Fast Forward Labs and the Machine Learning Landscape | 0 | 3 | 1 | 0 | Hilary Mason delivers a presentation on the transition from big data infrastructure to AI/machine learning capabilities. As this segment is a solo presentation monologue without host interaction, host expertise and pushback scores are zero. | |
| Bridging Communities and the Enterprise Innovation Gap | 0 | 4 | 1 | 0 | Hilary explains the innovation gaps between academia, startups, and enterprise companies, highlighting why off-the-shelf vendor products fail generic machine learning needs. Host metrics remain zero during this solo presentation segment. | |
| Discovery Process and Research Breakthroughs | 0 | 4 | 1 | 0 | Hilary outlines Fast Forward Labs' criteria for identifying applied research breakthroughs, focusing on cross-domain application and compute economics. The host is not present in this monologue segment. | |
| Technology Commoditization and Accessible Infrastructure | 0 | 4 | 1 | 0 | Hilary discusses technology commoditization using Hadoop as an example, alongside public data sources like Wikipedia underpinning commercial data science. Host participation is zero during the talk. | |
| Natural Language Generation and NYC Real Estate Prototype | 0 | 4 | 1 | 0 | Hilary details natural language generation capabilities and presents FFL's real estate prototype, pointing out how listings use words like cozy to mask tiny square footage. This is a monologue segment. | |
| Image Analysis and Machine Learning Failures | 0 | 4 | 1 | 0 | Hilary highlights rich media analysis and shares a case study where subway photos were classified as prisons due to ImageNet dataset gaps. Host scores are zero during the presentation. | |
| Rich Media Analytics, Summarization, and Review Mining | 0 | 4 | 1 | 0 | Hilary covers text summarization, review clustering, and probabilistic programming for Bayesian decision-making. Host metrics remain zero throughout the presentation monologue. |
Statements from this episode (15)
Mason: Machine learning startups struggle with innovation due to lack of data
“There are challenges for startups, because you don't have data.”
Mason: Building generic ML products is harder than solving single enterprise problems
“When somebody, when a vendor or a startup is going to build a product that solves your problem, They must solve a generic formulation of the problem. They have to solve everybody's version of your same problem. When you want to solve your problem, you just nee…”
Mason: Most tech breakthroughs apply existing research to entirely new domains
“It most often is something where there's been research in one domain that is applicable to another domain. So it's not necessarily something entirely new, but rather somebody Discovering something that may have existed for a while, but realizing that it's appl…”
Mason's 2016 prediction: Deep learning compute will be trivially affordable by 2018
“These days you can afford it and I expect in a year or two it'll be trivial to afford it.”
Mason: Open source is rapidly commoditizing machine learning capabilities
“We're seeing this in machine learning primarily in the open source world in an ongoing basis, almost something new every day.”
Mason: Wikipedia data secretly underpins almost every open machine learning API
“If you go back far enough in pretty much any sort of open API in machine learning, you can find Wikipedia in there somewhere.”
How the Associated Press used algorithmic systems to automate financial reporting
“They're working with a company called Automated Insights to write a bunch of finance stories. They actually have someone called Automation Editor whose job it is is to manage these algorithmic systems generating content.”
NYC apartments described as 'cozy' average 400 square feet smaller than normal
“If it says cozy, it is 400 square feet smaller than the average for that zip code.”
Mason: Natural language generation helps us comprehend data, not replace reporters
“The real impact here is not in replacing reporters but in helping people understand complex data.”
Mason: The ImageNet dataset completely lacked training data for NYC subways
“There were, was no training data in the ImageNet data set of the New York City subway system.”
Mason: Roughly 12 percent of daily Bitly links were unanalyzed rich media
“Around 12% of the links we saw on a daily basis were primarily media objects, and we had no insight into them.”
Mason: Real estate valuation is fundamentally an inference problem
“Real estate is a fantastic inference problem because the only way to know the value of a property is to sell it, but obviously we don't sell every property every day”
Mason: Deep learning completely removes the need for domain feature engineering
“In deep learning, you're not doing feature engineering anymore. Nobody cares about what you know about the domain.”
Mason: Sentiment analysis remains fundamentally unsolved due to lacking ground truth
“I think there are a couple of things that everyone thinks are solved problems that are absolutely not, and so something like sentiment analysis is something where we sort of take for granted that you can just plug into an API and get a number back, but this is…”
Mason: Publications reflect past research, while informal chats reveal future ideas
“And so I really believe if you want to know what someone did a year ago, you read their publication, and if you want to know what they're doing now, you watch them give a talk like this, and if you want to know the sort of wacky ideas they're thinking about, y…”