Steier: Modern data warehouses wouldn't exist if 1980s spreadsheets handled big data
Sandy Steier · Sandy Steier, 1010data // Data Driven NYC 24 // February 2014 (Hosted by FirstMark Capital) · Mar 3, 2014 · at 12:32
Sandy Steier, CEO of 1010data, presents a thought experiment at Data Driven NYC on how early software limitations shaped enterprise data architecture.
0:00 / 0:24exact quote · 24.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“If in the early eighties, You know, if we could go back and go into a time machine and deliver to the past a spreadsheet that worked on unlimited amounts of data, would things have happened the way they happened? You know, yes, somehow data governance would have to be taken care of. I'm, I don't have answers to all these things, but, you know, I'm not going to get into all the details. Somehow these things would have to be taken care of, but I guarantee that the data warehouse stack would not look like it looks today.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Sandy Steier
Steier: Big data discovery could replace the entire enterprise data stack
“I think that, in fact, data discovery, which is becoming more and more robust and taking over more and more of sort of the stack, I think it could take over the entire stack, or actually replace the stack.”
Steier: Data volume limits drove the split between spreadsheets and databases
“So the reason why you switched from spreadsheets to databases, which is a completely different kind of thing, right, is because of spreadsheets couldn't handle the larger amounts.”
Steier: Database engines are being partially replaced by Hadoop
“And in particular, this sort of, the database engine itself is being replaced to a certain degree with things like Hadoop.”
Steier: Data warehouses cannot run huge ad-hoc analyses on a whim
“The data warehouse is not capable of doing very large analyses for just, you know, on a whim by anybody who might happen to ask a question”
Steier: Adding 10 billion rows to a data warehouse takes a long time
“If I need to, if I have a new data set, somebody hands me ten billion rows of data, and I want to do an analysis on it, you know, to get a data warehouse up and running, or to add that to an existing data warehouse is a non-trivial thing to do. It takes a long…”