Jordan says personalization business models fail due to statistical limits
Michael Jordan · Michael Jordan · Jul 28, 2017 · at 2:53
UC Berkeley professor Michael Jordan addresses the gap between commercial expectations for big data personalization and the underlying statistical limits of sparse user data.
0:00 / 0:10exact quote · 10.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“A lot of these business models are failing. People actually can't personalize very well, and it's Because of statistical issues. You've got huge amounts of data about some people, and very little about lots of people, and you don't know how to transfer the statistics about, around that.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Michael Jordan
Jordan predicts principled personalized big data systems remain decades away
“So I think we're decades away from being able to do what this boss is asking us to do in some principle way. You can occasionally build a one-off system that does some of these things, but we're decades from having the real principles.”
Jordan says tech industry builds software without addressing underlying statistical issues
“We're building lots of software, hoping it works, and we're not really thinking about the harder intellectual issues at play.”
Jordan warns high-dimensional Bayesian inference is overly sensitive to unknown priors
“And a lot of times you have no idea what the prior should be. You don't know what the tails should be in particular. And you're in high dimensions, you really have no idea how the tail behavior should be. And the whole inference is highly sensitive to the tail…”
Jordan notes core statistical decision theory ignores computational runtime
“If you look at core statistical theory, it's statistical decision theory, I teach it all the time, you never see the word run time.”
Jordan argues differential privacy is rarely analyzed for statistical inference
“Differential privacy has mostly not been thought about inferentially”
Jordan argues meaningful real-world decisions require statistical error bars
“And real life decisions that mean something, you need error bars. If they don't mean anything, you're just trying to serve customers and hope that everybody comes to your website, you know, I don't know, who cares? But in real life, you need error bars.”