StarZero All shows Try the podcast clipper →

Mar 3, 2014 · 20m · mad

Sandy Steier, 1010data // Data Driven NYC 24 // February 2014 (Hosted by FirstMark Capital)

Sandy Steier · 13m spoken Owen Simpson · 3m spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

At Data Driven NYC, 1010data CEO Sandy Steier outlines the historical split between enterprise data warehouses and business spreadsheets, demonstrating how modern big data discovery platforms can merge these paradigms to allow instant self-service analysis on tens of billions of rows.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 1.5% of the talking time here. How this is scored →

Matt as informed peer 0.0 Guest teaching 4.0 Guest disagreement 1.3 Matt pushing back 0.0
05100:0010:0020:000:37–4:53 · Matt as informed peer 0/10 History and Evolution of Data Analysis Sandy Steier gives a solo presentation detailing the evolution of data analysis from IBM punch cards to spreadsheets and relational databases. The host does not participate during this monologue, requiring host scores to be zero.4:53–9:07 · Matt as informed peer 0/10 Current Workflow Challenges and Data Warehouse Limitations Steier outlines the friction between IT data warehouses and business analysts, pointing out inefficiencies and long wait times. Host-side engagement is non-existent as this remains a presentation monologue.9:07–13:04 · Matt as informed peer 0/10 Root Cause Analysis and Spreadsheet Thought Experiment Steier proposes a thought experiment asking why spreadsheets were abandoned for databases simply due to scale, educating the audience on foundational architecture assumptions. The host is passive during the presentation.13:04–15:46 · Matt as informed peer 0/10 Reimagining the Future with Big Data Discovery Steier concludes his visionary argument that big data discovery tools will replace traditional stack layers before handing off to Owen Simpson for a live product demonstration. The host is inactive during the talk.0:37–4:53 · Guest teaching 4/10 History and Evolution of Data Analysis Sandy Steier gives a solo presentation detailing the evolution of data analysis from IBM punch cards to spreadsheets and relational databases. The host does not participate during this monologue, requiring host scores to be zero.4:53–9:07 · Guest teaching 4/10 Current Workflow Challenges and Data Warehouse Limitations Steier outlines the friction between IT data warehouses and business analysts, pointing out inefficiencies and long wait times. Host-side engagement is non-existent as this remains a presentation monologue.9:07–13:04 · Guest teaching 5/10 Root Cause Analysis and Spreadsheet Thought Experiment Steier proposes a thought experiment asking why spreadsheets were abandoned for databases simply due to scale, educating the audience on foundational architecture assumptions. The host is passive during the presentation.13:04–15:46 · Guest teaching 3/10 Reimagining the Future with Big Data Discovery Steier concludes his visionary argument that big data discovery tools will replace traditional stack layers before handing off to Owen Simpson for a live product demonstration. The host is inactive during the talk.0:37–4:53 · Guest disagreement 1/10 History and Evolution of Data Analysis Sandy Steier gives a solo presentation detailing the evolution of data analysis from IBM punch cards to spreadsheets and relational databases. The host does not participate during this monologue, requiring host scores to be zero.4:53–9:07 · Guest disagreement 2/10 Current Workflow Challenges and Data Warehouse Limitations Steier outlines the friction between IT data warehouses and business analysts, pointing out inefficiencies and long wait times. Host-side engagement is non-existent as this remains a presentation monologue.9:07–13:04 · Guest disagreement 1/10 Root Cause Analysis and Spreadsheet Thought Experiment Steier proposes a thought experiment asking why spreadsheets were abandoned for databases simply due to scale, educating the audience on foundational architecture assumptions. The host is passive during the presentation.13:04–15:46 · Guest disagreement 1/10 Reimagining the Future with Big Data Discovery Steier concludes his visionary argument that big data discovery tools will replace traditional stack layers before handing off to Owen Simpson for a live product demonstration. The host is inactive during the talk.0:37–4:53 · Matt pushing back 0/10 History and Evolution of Data Analysis Sandy Steier gives a solo presentation detailing the evolution of data analysis from IBM punch cards to spreadsheets and relational databases. The host does not participate during this monologue, requiring host scores to be zero.4:53–9:07 · Matt pushing back 0/10 Current Workflow Challenges and Data Warehouse Limitations Steier outlines the friction between IT data warehouses and business analysts, pointing out inefficiencies and long wait times. Host-side engagement is non-existent as this remains a presentation monologue.9:07–13:04 · Matt pushing back 0/10 Root Cause Analysis and Spreadsheet Thought Experiment Steier proposes a thought experiment asking why spreadsheets were abandoned for databases simply due to scale, educating the audience on foundational architecture assumptions. The host is passive during the presentation.13:04–15:46 · Matt pushing back 0/10 Reimagining the Future with Big Data Discovery Steier concludes his visionary argument that big data discovery tools will replace traditional stack layers before handing off to Owen Simpson for a live product demonstration. The host is inactive during the talk.

speaking balance: gold is Matt, purple is the guest (3 minute bins)

0:00 · Matt 5.4% · guest 94.6%0:00 · Matt 5.4% · guest 94.6%3:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%15:00 · Matt 0% · guest 100%15:00 · Matt 0% · guest 100%18:00 · Matt 6.9% · guest 93.1%18:00 · Matt 6.9% · guest 93.1%
Sharpest disagreement ▶ 5:20 Critique of data warehouse stack inefficiency

Steier forcefully highlights how traditional data warehouse workflows take weeks and frustrate business analysts.

 Hardest push from Matt ▶ 19:55 Host cuts off audience Q&A due to time limits

Matt Turck steps in at the end of the presentation to state that time has run out for questions.

 Biggest teaching moment ▶ 10:15 Reframing database history through spreadsheet scale

Steier challenges traditional computer science norms by arguing that data warehouses only exist because 1980s spreadsheets could not handle row scale.

 Matt holds his own ▶ 0:21 Host manages audio glitch at the start

Matt Turck briefly steps in to troubleshoot the guest's microphone setup before the talk begins.
the scores for every segment, with the reasoning behind each
ChapterTopicMatt as informed peerGuest teachingGuest disagreementMatt pushing backWhy
History and Evolution of Data Analysis 0410 Sandy Steier gives a solo presentation detailing the evolution of data analysis from IBM punch cards to spreadsheets and relational databases. The host does not participate during this monologue, requiring host scores to be zero.
Current Workflow Challenges and Data Warehouse Limitations 0420 Steier outlines the friction between IT data warehouses and business analysts, pointing out inefficiencies and long wait times. Host-side engagement is non-existent as this remains a presentation monologue.
Root Cause Analysis and Spreadsheet Thought Experiment 0510 Steier proposes a thought experiment asking why spreadsheets were abandoned for databases simply due to scale, educating the audience on foundational architecture assumptions. The host is passive during the presentation.
Reimagining the Future with Big Data Discovery 0310 Steier concludes his visionary argument that big data discovery tools will replace traditional stack layers before handing off to Owen Simpson for a live product demonstration. The host is inactive during the talk.

Statements from this episode (8)

Assertion Not checkable as stated
Steier: Database engines are being partially replaced by Hadoop
“And in particular, this sort of, the database engine itself is being replaced to a certain degree with things like Hadoop.”
Sandy Steier Mar 3, 2014 ▶ 3:43
Assertion Partly supported
Steier: The term 'data discovery' is only a few years old
“The term itself, data discovery, is only a couple years old, or so.”
Sandy Steier Mar 3, 2014 ▶ 4:15
Assertion Not checkable as stated
Steier: Data warehouses cannot run huge ad-hoc analyses on a whim
“The data warehouse is not capable of doing very large analyses for just, you know, on a whim by anybody who might happen to ask a question”
Sandy Steier Mar 3, 2014 ▶ 5:09
Assertion Not checkable as stated
Steier: Adding 10 billion rows to a data warehouse takes a long time
“If I need to, if I have a new data set, somebody hands me ten billion rows of data, and I want to do an analysis on it, you know, to get a data warehouse up and running, or to add that to an existing data warehouse is a non-trivial thing to do. It takes a long…”
Sandy Steier Mar 3, 2014 ▶ 7:28
Insight
Steier: Data volume limits drove the split between spreadsheets and databases
“So the reason why you switched from spreadsheets to databases, which is a completely different kind of thing, right, is because of spreadsheets couldn't handle the larger amounts.”
Sandy Steier Mar 3, 2014 ▶ 11:34
What-if
Steier: Modern data warehouses wouldn't exist if 1980s spreadsheets handled big data
“If in the early eighties, You know, if we could go back and go into a time machine and deliver to the past a spreadsheet that worked on unlimited amounts of data, would things have happened the way they happened? You know, yes, somehow data governance would ha…”
Sandy Steier Mar 3, 2014 ▶ 12:32
Prediction Not checkable as stated
Steier: Big data discovery could replace the entire enterprise data stack
“I think that, in fact, data discovery, which is becoming more and more robust and taking over more and more of sort of the stack, I think it could take over the entire stack, or actually replace the stack.”
Sandy Steier Mar 3, 2014 ▶ 14:18
Disclosure
1010data holds complete U.S. stock exchange trade history back to 2003
“They house data from all U.S. Exchanges, all trades that are executed every day throughout the U.S. And we have data dating back to 2003.”
Owen Simpson Mar 3, 2014 ▶ 16:07
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more