The Exchanges

Q And then you, you found your way into Shopify, which is actually, by the way, interesting for anyone, um, that's, uh, a startup in trying to recruit, uh, top data scientists. You, you found your way to Shopify through a hackathon?

A Yeah, so there was this hackathon called, uh, Random Hacks of Kindness, and, uh, basically companies, uh, or people get together to solve problems for non-profit companies where they don't have a large tech team. So I went to the Shopify office, they were holding the hackathon, and I started talking to people there, and they said, yeah, we are growing a data team, and why don't you, They said, why don't you come in for a talk? So I went in for a talk, which was the interview, and then, uh, then I got an offer, so I never actually submitted a resume, and I think that's, like, one of the strengths of Shopify in hiring is that it, we don't go super conventional ways. We just try to get to know people, know about their experiences, and excite them about problems we are trying to solve.

AI assessment note: “Yeah, so there was this hackathon called, uh, Random Hacks of Kindness”

Q to the audience, um, to, you know, take a lot of what you said and, like, putting it back into your framework, um, something you, you mentioned in some of your talks, the, you know, Monica, Monica Rogatti's, um, Pyramid. Uh, do you, do you want to explain why, what that is, uh, and then how that relates to Shopify's journey, um, through two data science and, and machine learning?

A Uh, for sure. So Monica Ruggari, who used to work in, I think, Fitbit and Jawbone for years, she has this great, uh, pyramid of data science hierarchy of needs, and it starts with, first of all, you have to do a data acquisition, so you do eventing, you do data collection, and once you have the data, you have to think about, like, how are you going to move it from one location, so either from operational systems or API endpoints to your data lake, um, and then pipelines for that movement and how resilient they are. Once you have that, then you can have reports and dashboards to sort of look at what happened in the business and be able to explain them. Um, then on top of that, you can have things that are, like, maybe closer to causal inference, where you do deeper statistical analysis. Then you can do A-B testing, and at the top of the pyramid is, uh, machine learning and AI. And I don't think they mean top as, like, the best, but it's, like, you have to go through this to be able to get to the top. And there has been times that even us, we've sort of, like, jumped to the top Uh, top in one of the areas, and you pay the technical debt that you did not do the work in the beginning. So, in the beginning when I joined Shopify, we were mostly trying to make sure we are collecting everything when we need it, because, you know, you're startup, you're moving fast, so data collection i…

AI assessment note: “when I joined Shopify, we were mostly trying to make sure we are collecting everything”

Q Great. All right, last question from me. Um, predictions, uh, for the world of, uh, AI, um, you know, one, two, three, five, whatever predictions, uh, what, what's, uh, what are some of the interesting things happening, whether the big things are very, you know, practical, um, things that are gonna happen in the next three to five years?

A Um, my predictions are a bit hopeful, so I hope also things go that way. Uh, one of them is, uh, the research and work around, uh, sort of fair, accountable, transparent algorithms become more commonplace, uh, because I think, uh, users like ourselves are, um, becoming more and more educated about their data and how these data products impact their life. So I really want that research to leave sort of just being academic and impact how we build products, how we collect data from our users and shape their experiences. The other one that I really want to see is this stop to sort of like human versus AI and human with AI. So I think, uh, and I think as, uh, people talked about it before me is that, uh, the more we can play to each other's strengths, the better. Uh, so I think like the more we can even build tools for combining human intelligence with like artificial intelligence, the more successful it would be. And the last one, I think, uh, there's a lot of hype and focus and good work in machine learning. But I do think if we take a step back and look at causal inference, we're gonna actually solve a lot of problems with just that arm. So I hope, uh, over the next few years, like, people invest in that, people learn from that, uh, I think social sciences have solved so many of these problems in the context of the past, and that would, uh, also stop some of these, uh, things tha…

AI assessment note: “These are my three hopes slash predictions.”

Q Great. And the next project was anti-fraud?

A Yes. Uh, so then after that, so I think it was like this pivotal moment in Shopify where we, before that we were using data for making data informed decisions about what to build, where to invest, all of those things. But I think that was a moment where we said, okay, we can actually use data to create experiences. So capital was the first one. Then we said, okay, uh, the experience of fraud for, for a small merchant or a big one is not a pleasant one because, uh, the merchant would lose the product, they would lose the money, and they would lose the trust in the buyers. And if you think of, like, a small merchant just starting, uh, that can hit them in their cash flow pretty hard. For large merchants also, they don't want to, like, go and review thousands of orders, and they don't necessarily have the experience. So it's okay, that's a problem that's worth it for us to solve at scale. Um, so we started actually, again, very basic, because it, for this problem especially, it wasn't just a model that was important. We had to be able to run this analysis in real time, because as soon as the order comes through, Uh, you want to be able to say how likely it is to be fraudulent so that the merchant can decide to fulfill it or not, right? Because if it's fraudulent, you don't want to send them the goods. So it meant that, uh, it has to be scalable to the scale of checkout. So at, uh,…

AI assessment note: “Yes. Uh, so then after that”

Q And what else? People work in, like, live in Jupyter notebooks, or?

A Uh, ok, so I can go a bit in that. So one thing we invested heavily in through, like, process of IPO and going forward is that we realized as a company, it's amazing that we have all of this data. It's amazing that we have this open data policy where everyone can access data and answer their questions. But the moment the consistency goes away across different answers for the same question, the trust in data goes away. So we invested, we invested a lot in making sure we have very solid data pipelines, For, like, building our ETL using Spark Python in-house language, um, and then for, so it depends on the kind of analysis. Like, we, we have this, uh, we use mode analytics for, like, SQL. Uh, we do Jupyter Notebooks for ad hoc or one-off analysis, uh, but for machine learning and things like that, we sort of run them through the same, uh, parts of our ETL platform that uses, uh, Spark, and yeah. So we do, like, streaming, things like that, but within the, the platform we have.

AI assessment note: “we do Jupyter Notebooks for ad hoc or one-off analysis”

Q Have you retired machine learning models because the experience turned out to not be as great?

A We have not shipped models, so we have this, like, two-week cycle, uh, where we put new features, we look at the lift, and we do a lot of backtesting, uh, to see, so backtesting is very common in finance, uh, and you run the model basically saying, like, what would have happened in production over the last six months if I had run this thing? Uh, and we, there has been time where we've seen, okay, this model is really good for a select slice of the merchants, but for everybody else, it's not gonna make their experience better, or it's gonna cause confusion, so we actually Don't ship it. We don't bring it to merchants. We learn what we can learn and move on to bringing next to the features and models.

AI assessment note: “We have not shipped models... we actually Don't ship it. We don't bring it”

Q Okay. And, um, let's talk about maybe the, we'll get into the process and all those things, but maybe the, the, the tools, the tech stack, what do data scientists work on in that sort of day to day?

A Uh, for sure. So as I joined, we were actually going through this, like, um, changing of our data platform, which is funny, because when they hired me, they said, you can join now, or you can join in three months when we have, uh, rebuilt our data warehouse. Yeah, that took two years. And I'm glad I joined at the time I joined. So, um, we basically moved from, uh, using Vertica, uh, to building a ETL tool in-house using, uh, Spark, uh, and, uh, Python. Um, and then, uh, right now, as of last year, we've been in, uh, GCS, Google Cloud, so, uh, and then in terms of, uh, sort of query layer, we use things like, uh, Presto heavily, a little bit of Redshift, um, and then, like, different visualization tools, uh,

AI assessment note: “we basically moved from, uh, using Vertica, uh, to building a ETL tool in-house”

Q And so, um, Shopify Capital, um, the anti-fraud, now you've built a whole portfolio of, uh, data science and machine learning driven products. What, what, what else have you guys built over the years? And, and, and also, how, how do you go about selecting the next thing and then getting it through?

A Maybe I start with how we go about first, because I think that's the key. Otherwise it would look like a random selection of things. Uh, so the way we go about it is that we are always like product and merchant focus. So we say, okay, this is a product we are trying to, this is a problem we are trying to solve for merchant in this domain. Uh, let's first understand the problem. Like what are they trying to do or what the buyer is trying to do? Learn a bit. And if we can, we would use the machine learning. So I always tell the team is that, uh, that data is very powerful. Machine learning is very powerful, but at the end of the day is a tool. Like if you use it properly, it's useful. If you just want to, if you use your axe for everything, it's not going to be a good use case. So, uh, one of the things we realized is that we can help merchants a lot in setting up their store or knowing what's the next best action to take. So when they log into their Shopify store, uh, in the admin, they see a bunch of, like, home cards that tells them, uh, do this or, like, uh, market for this product or make this change in your theme. And those are all driven by machine learning, um, And we, we track it by thinking like, okay, how often do merchants actually do the things we tell them, rather than like, oh, we think this is a really cool algorithm to put out there. Uh, we have a marketing, uh, …

AI assessment note: “Maybe I start with how we go about first, because I think that's the key.”

Q Very good. So, uh, why don't we start with your journey. Uh, so you're born in Iran, I believe, to a professor, father, and mother, is that correct? Tell us, how did you, how did you end up in your current position?

A Uh, I think, uh, as I was growing up, I think one of the luxuries I had was that, uh, my parents always, like, gamified math and computer science, and one of my earliest interactions was that, uh, actually let me ask a question, because everyone was asking about modern technologies, so I'm gonna ask how many people in the audience know what's a punch card? How many have ever seen it being used? Awesome. Ok, so my mom was, uh, was doing this thing, and I was like, oh, are you doing crafts? You're making holes in this, uh, cards, and she said, no, I'm actually telling a computer to do something, and as a four year old, that was just the most fascinating thing, so I don't think I ever learned math or computer science to be good at it, but it was, uh, what it enabled, uh, us to do, and I think that's how I got started, um, and then

AI assessment note: “I think that's how I got started”