Machine Learning Systems

“Machine learning systems it's easy to get to an 80% solution, but all the values in that marginal 15 or 18 to get to 95 to 98% accuracy.”

“It might be giving, it might have internally created a category and given those categories of people, lower risk and lower, lower insurance prices, and those might be women, but it might, it wouldn't have a concept of women and you might not know if it's doing…”

“These systems are basically pattern matching pattern recognition systems at a very, very simplistic level. There might be stuff in the pattern that you want it to use. There might be stuff in the pattern. There might be pattern that you didn't know was there. …”

“Fundamentally, the thing we are actually optimizing is actually not that different. It's always about data movement across essentially all the stacks, right? So when you do distributed, like computing, it's about communication across different machines. When y…”

“Code drift there, which arguably is the number one, is the only trigger for most web apps. You know, your code drift, someone adds something, you rebuild your model, which is your web application. But actually what you care about in these new systems is data d…”

“Well, yeah, the intuition for how to configure these systems does not transfer, which is why you need to retune, re-optimize, and reconfigure these systems to make sure they're maximizing that business value.”

“And to my knowledge, it's the first time that like the creative output of a machine learning system has been included in an audio book.”

“You actually try to make that core be as small as possible because machine learning is really hard. It's a lot of compute. It's really hard to tell what's going on there. And so you want it to be as simple as possible, but then you surround it by as much engin…”

“And like the way that, that I now think about machine learning systems is that they're really a way to make the leverage of human programmers go way up.”

“Everybody always assumes that like, okay, if it can be automated, it definitely will be automated, right? But that's like a fallacy, because in certain cases, like, you may really worry about the security of your systems, right?”

“This emerging research right now, which is using machine learning to train machine learning systems, raises, like, this meta level, where, like, right now there's a lot of handwork that goes into building a model so it learns the right representations, but, li…”

“So what happens whenever you deploy a machine learning system is that the people who are being modeled Change their behavior in response to the system. Sometimes in benign ways, but sometimes in adversarial ways.”

“And once, for example, these machine learning systems become more widely available as they are becoming, people will start using them and the doctors will be forced to catch up.”

“So, at least for you know, I think the next five to 10 years, there's going to be heavy involvement of humans in machine learning systems.”

“Over time, if you build the appropriate feedback loops into your systems, the system itself will wind up learning from those processes and get better and better, so you can actually start automating those processes.”