The Exchanges

Q Well, I want to start. How did you make your way into the world of product? It's an interesting world. What was your entry point?

A You know, I'm a civil engineer by training, environmental civil, and then I became a nuclear engineer. So it's not an obvious route, but I ended up, um, at the startup, Plumtree Software, and, uh, first running QA, then running engineering. This guy that ran product, Phil Sofer, he was like, A very good friend of mine, and every day I told him how he was doing his job wrong. I was the receiving end, building stuff, right? So I just told him this every day, and then one day he decided to move on to leave Plumtree, and he told the CEO I had to run product to, like, get back at me, right? I did it. I didn't want to do it, you know. I was like, My job's too important doing engineering. You know, I can't give that up. But once I got into it, it was incredible.

AI assessment note: “he told the CEO I had to run product”

Q I, I, I totally agree. Is speed everything? Everyone's like, the only thing that matters is speed and velocity. You illustrated two very different, uh, types of movement there. Is speed everything, or actually do people overemphasize the importance of speed?

A I mean, speed of what? Like, speed of learning is everything. But that doesn't mean shipping. That doesn't mean, you know, people talk about one-way doors, Two-way doors, type one, type two decisions, and all this stuff. Like, the key is maximizing your learning curve, and there's so many clever ways to do that, and a lot of it feels slow. You want to know Why you are doing the next step, you know, before you do it. But then sometimes you can't figure it out that way, and you just need to try. And that's when, you know, ship things that you might throw away and do it, you know, don't scale prematurely, don't over architect it, get the learning in.

AI assessment note: “speed of learning is everything. But that doesn't mean shipping.”

Q Do you not think that they use data as a crutch to kind of lean on because they don't actually know?

A It's nice when you- I think that's it. Because here's the thing. I'm just as bad at this as everyone else. Like, I have these dashboards. I build a lot of dashboards of a lot of metrics. And if there's something I don't understand, I like do data science until I figure it out. Um, and then I look at these all the time. Like, this morning I got up, I looked at some dashboards. Am I going to do anything with that? No, it's a fucking waste of time. But it hits that, it's like infinite scroll. I want to see if the next bar is green or not. I have like this dashboard where if it's a record day for any sub-segment, Like a record Tuesday. Because, you know, there's obviously the big, the, the, the curve on the week. It's green. And I call it record book for all these different parts of our product. I look at every day. I get, it's green. It's green. We're rocking it. But, like, I'm not going to do anything with that. So it's a waste of time, and think about the time people spend. A lot of times you don't have a good telemetry in place, you don't have good BI systems, so there's humans spending time counting shit to put the number in to send out the email with all the numbers so people see it, and is that adding value? That's the problem. The human energy waste that goes in to data-driven product management When it's not becoming actionable.

AI assessment note: “I think that's it. Because here's the thing. I'm just as bad at this”

Q Possible if everyone was inside my brain. It's like reality distortion fields. What's different seeing good reality distortion and bad reality?

A Every great leader in startups, um, like, you're, you're bending reality, you know? You're, you're, you're creating things that seem Impossible. So you have to enroll people in that, and once they're enrolled, if you think something's impossible, but you convince three friends, then the four of you think something's impossible, but the world, reality, thinks it's impossible. That's reality distortion. But you enroll people in it so they can hunker down and prove everybody wrong and accomplish it. As organizations grow and, you know, leaders have ego, um, you can Believe yourself and not be truth-seeking. And then you can do your company a lot of damage, and you can do people a lot. Think about how many people that have been traumatized by startups, you know? It was kind of like, we want to accomplish this by any means necessary, because it's the right thing to do, and you start to see less and less what you're compromising to do that. And so, if you get caught up in your own reality distortion field, like, it might be impossible. So you've convinced all these people that something impossible is possible, and they're going to waste a lot of, you know, their career chasing you on it because you're so convincing. That's when it gets, it gets, you know, a dark side.

AI assessment note: “you can Believe yourself and not be truth-seeking. And then you can do your company”

Q Do you not think that they use data as a crutch to kind of lean on because they don't actually know?

A It's nice when you- I think that's it. Because here's the thing. I'm just as bad at this as everyone else. Like, I have these dashboards. I build a lot of dashboards of a lot of metrics. And if there's something I don't understand, I like do data science until I figure it out. Um, and then I look at these all the time. Like, this morning I got up, I looked at some dashboards. Am I going to do anything with that? No, it's a fucking waste of time. But it hits that, it's like infinite scroll. I want to see if the next bar is green or not. I have like this dashboard where if it's a record day for any sub-segment, Like a record Tuesday. Because, you know, there's obviously the big, the, the, the curve on the week. It's green. And I call it record book for all these different parts of our product. I look at every day. I get, it's green. It's green. We're rocking it. But, like, I'm not going to do anything with that. So it's a waste of time, and think about the time people spend. A lot of times you don't have a good telemetry in place, you don't have good BI systems, so there's humans spending time counting shit to put the number in to send out the email with all the numbers so people see it, and is that adding value? That's the problem. The human energy waste that goes in to data-driven product management When it's not becoming actionable.

AI assessment note: “I think that's it. Because here's the thing. I'm just as bad at this”

Q I, I, I totally agree. Is speed everything? Everyone's like, the only thing that matters is speed and velocity. You illustrated two very different, uh, types of movement there. Is speed everything, or actually do people overemphasize the importance of speed?

A I mean, speed of what? Like, speed of learning is everything. But that doesn't mean shipping. That doesn't mean, you know, people talk about one-way doors, Two-way doors, type one, type two decisions, and all this stuff. Like, the key is maximizing your learning curve, and there's so many clever ways to do that, and a lot of it feels slow. You want to know Why you are doing the next step, you know, before you do it. But then sometimes you can't figure it out that way, and you just need to try. And that's when, you know, ship things that you might throw away and do it, you know, don't scale prematurely, don't over architect it, get the learning in.

AI assessment note: “speed of learning is everything. But that doesn't mean shipping.”

Q Before we dive into kind of whether it's good or bad, what does that actually mean, being a data-driven product manager? When I saw this, I was like, huh, what, what is that?

A So, you know, Databricks used to have this value called data-driven. Um, where we wanted everybody to be data driven, but it has a dark side because you can weaponize data, right? You can argue any point of view with any set of data if you're clever enough with numbers, right? So we, we modified that to be truth seeking because the point of being data driven was be truth seeking, but truth seeking defangs the dark side of data driven, right? So let's say you're going to build a product. You want to like ship early, ship often, but how can you figure out the early signal If it's not revenue based, cause maybe you're not charging it of whether or not it's working. Right. And you come up with a KPI and you can set goals and stuff like that. And you track that. And we could talk about that for days, you know, like input metrics versus output metrics and things like that. As soon as you start like reporting on it, it becomes a thing. It becomes the goal. The data in your progress serves one purpose, one purpose only, which is to figure out your blind spots and where to ask questions. The goal is not hitting your target. The goal is seeing where you're varying or going way above your target. Why? And then you can dig in and figure out if you need to course correct or not. A lot of times the right thing to do is change the target. The target was wrong. Like you came up with a target w…

AI assessment note: “The data in your progress serves one purpose... to figure out your blind spots”

Q Before we dive into kind of whether it's good or bad, what does that actually mean, being a data-driven product manager? When I saw this, I was like, huh, what, what is that?

A So, you know, Databricks used to have this value called data-driven. Um, where we wanted everybody to be data driven, but it has a dark side because you can weaponize data, right? You can argue any point of view with any set of data if you're clever enough with numbers, right? So we, we modified that to be truth seeking because the point of being data driven was be truth seeking, but truth seeking defangs the dark side of data driven, right? So let's say you're going to build a product. You want to like ship early, ship often, but how can you figure out the early signal If it's not revenue based, cause maybe you're not charging it of whether or not it's working. Right. And you come up with a KPI and you can set goals and stuff like that. And you track that. And we could talk about that for days, you know, like input metrics versus output metrics and things like that. As soon as you start like reporting on it, it becomes a thing. It becomes the goal. The data in your progress serves one purpose, one purpose only, which is to figure out your blind spots and where to ask questions. The goal is not hitting your target. The goal is seeing where you're varying or going way above your target. Why? And then you can dig in and figure out if you need to course correct or not. A lot of times the right thing to do is change the target. The target was wrong. Like you came up with a target w…

AI assessment note: “The data in your progress serves one purpose... to figure out your blind spots”

Q Have you always felt like that? Often what I see with product leaders is they tend to Find that science leads the way in that early product.

A I was, I was just going to go there. Like I have created so many detailed frameworks, you know, and it's basically like I have mathematical proof that this is what we should do. I refer to things with the wrong names all the time because of my like short term memory. And the, there's something like the fallacy of misplaced concreteness. It's an economic theory. And the, the gist of it is the more detail you see, The more correct you think something is. You can have this incredible financial model. Like, you know, people do financial models and spreadsheets all the time. You can have like a very simple model, or you can have one billions of pages, and you're like, they put so much work into that. They're really good. But it's bullshit. A lot of it is using your collective experience, and there's a lot of science and math in that, to find holes in the strategy, rather than, like, because humans are not mathematically definable, and products are for humans. So, over rigor is just waste in the system, I think.

AI assessment note: “I was, I was just going to go there. Like I have created so many detailed frameworks”

Q Always. Um, How do you structure the hiring process for new additions to your product teams in terms of the structure? Is there a...

A So the first, the first conversation I have with someone, it's the same as a customer interview or, or talking to someone in another function to move into product. So I want to understand impact you've had, and then you'll say something, and until I think I could do it, I keep asking questions. But you do that until you find someone's floor. Right? So, I have an engineering background, or at least a civil engineering background. I can decompose systems, you know. Um, and a lot of product managers don't know how their shit works. Out, you know, because Databricks are very tech, I, in some consumer companies that might be okay, but in enterprise software, you got to know how that shit works, because otherwise you'll give, make bad decisions because you won't understand the dependencies, right? We talked about how, you know, someone that was at Google during the aughts, whatever they did, the numbers went up, you know? Um, but was it causal is the question. So trying to understand how they dealt with a difficult situation, is it other people's fault? Do they take responsibility? How did they get out of it? Were other people damaged in the way they got out of it?

AI assessment note: “the first conversation I have with someone, it's the same as a customer interview”

Q Have you always felt like that? Often what I see with product leaders is they tend to Find that science leads the way in that early product.

A I was, I was just going to go there. Like I have created so many detailed frameworks, you know, and it's basically like I have mathematical proof that this is what we should do. I refer to things with the wrong names all the time because of my like short term memory. And the, there's something like the fallacy of misplaced concreteness. It's an economic theory. And the, the gist of it is the more detail you see, The more correct you think something is. You can have this incredible financial model. Like, you know, people do financial models and spreadsheets all the time. You can have like a very simple model, or you can have one billions of pages, and you're like, they put so much work into that. They're really good. But it's bullshit. A lot of it is using your collective experience, and there's a lot of science and math in that, to find holes in the strategy, rather than, like, because humans are not mathematically definable, and products are for humans. So, over rigor is just waste in the system, I think.

AI assessment note: “I have created so many detailed frameworks, you know, and it's basically like”

Q Always. Um, How do you structure the hiring process for new additions to your product teams in terms of the structure? Is there a...

A So the first, the first conversation I have with someone, it's the same as a customer interview or, or talking to someone in another function to move into product. So I want to understand impact you've had, and then you'll say something, and until I think I could do it, I keep asking questions. But you do that until you find someone's floor. Right? So, I have an engineering background, or at least a civil engineering background. I can decompose systems, you know. Um, and a lot of product managers don't know how their shit works. Out, you know, because Databricks are very tech, I, in some consumer companies that might be okay, but in enterprise software, you got to know how that shit works, because otherwise you'll give, make bad decisions because you won't understand the dependencies, right? We talked about how, you know, someone that was at Google during the aughts, whatever they did, the numbers went up, you know? Um, but was it causal is the question. So trying to understand how they dealt with a difficult situation, is it other people's fault? Do they take responsibility? How did they get out of it? Were other people damaged in the way they got out of it?

AI assessment note: “the first conversation I have with someone, it's the same as a customer interview”

Q How do you, that's really interesting. So they say what you want them to say. How do you actually do customer questioning? Well, I find a lot of people lead the witness.

A I mean, look, I'm, I'm a human, so I'm terrible at customer questioning because it's impossible not to lead the witness. It's really, really hard. So I, I oftentimes after, you know, I've been doing this for a long time. I like to have other people that I know on, on calls. And then I ask them for like notes. One of the, the, you know, Ali's cultural values, but I, I've kind of internalized it even, you know, pre, pre data breaks is truth seeking. Like you want to get at the truth. You don't want to, you know, there's all the biases, confirmation bias, blah, blah, blah, blah, blah. The problem is open-ended questions. Oftentimes you can't get to where you want to go. Right? So then it's just like open-ish questions that are kind of leading. You know, how would it feel if It happened this way instead. Rather than like, let me talk at you and show you a demo, and this is how it should be.

AI assessment note: “open-ish questions that are kind of leading. You know, how would it feel”

Q Product reviews that, you know, I'm not in product, obviously. They're kind of hailed as this kind of black box. How often Often do you do product reviews? Who's invited? Who sets the agenda? Can you just walk me through it?

A Well, like, ok. So, it's very different from, like, a brand new thing, a brand new feature, brand new product. In short, there's like, it's like fractal. So, I use a product all the time. I'm reviewing the product when I'm using it, like, continuously. And so, getting into, like, the preview, we have this dog food system and stuff like that. That's kind of like, uh, uh, you know, Asynchronous product review. But then, you know, in the classic scrum sprint methodology, you, you demo every Friday, you know. You want the core group to be seeing things as it's being built, because you can't waste time, and if it's going the wrong direction, you need to know soon, right? But then, you should have, like, continuous product reviews for the core team, and then, you know, monthly or quarterly, For management. But the danger, the danger is, like, the seagull management. Like, everybody-

AI assessment note: “continuous product reviews for the core team, and then, you know, monthly or quarterly”

Q Possible if everyone was inside my brain. It's like reality distortion fields. What's different seeing good reality distortion and bad reality?

A Every great leader in startups, um, like, you're, you're bending reality, you know? You're, you're, you're creating things that seem Impossible. So you have to enroll people in that, and once they're enrolled, if you think something's impossible, but you convince three friends, then the four of you think something's impossible, but the world, reality, thinks it's impossible. That's reality distortion. But you enroll people in it so they can hunker down and prove everybody wrong and accomplish it. As organizations grow and, you know, leaders have ego, um, you can Believe yourself and not be truth-seeking. And then you can do your company a lot of damage, and you can do people a lot. Think about how many people that have been traumatized by startups, you know? It was kind of like, we want to accomplish this by any means necessary, because it's the right thing to do, and you start to see less and less what you're compromising to do that. And so, if you get caught up in your own reality distortion field, like, it might be impossible. So you've convinced all these people that something impossible is possible, and they're going to waste a lot of, you know, their career chasing you on it because you're so convincing. That's when it gets, it gets, you know, a dark side.

AI assessment note: “That's when it gets, it gets, you know, a dark side.”

Q How do you, that's really interesting. So they say what you want them to say. How do you actually do customer questioning? Well, I find a lot of people lead the witness.

A I mean, look, I'm, I'm a human, so I'm terrible at customer questioning because it's impossible not to lead the witness. It's really, really hard. So I, I oftentimes after, you know, I've been doing this for a long time. I like to have other people that I know on, on calls. And then I ask them for like notes. One of the, the, you know, Ali's cultural values, but I, I've kind of internalized it even, you know, pre, pre data breaks is truth seeking. Like you want to get at the truth. You don't want to, you know, there's all the biases, confirmation bias, blah, blah, blah, blah, blah. The problem is open-ended questions. Oftentimes you can't get to where you want to go. Right? So then it's just like open-ish questions that are kind of leading. You know, how would it feel if It happened this way instead. Rather than like, let me talk at you and show you a demo, and this is how it should be.

AI assessment note: “open-ish questions that are kind of leading. You know, how would it feel”

Q When you change your destination, or you realize that actually what you thought it would be, it's no longer, how do you communicate that effectively without being like a flimsy leader? Do you know what I mean? It's like, hey, I know I said that we were doing this, but like, I was wrong.

A It's so ingrained in being a human, you know? Yeah. So, and it ties to the thing about like, willing something into existence. Like, I think, and this, this will sound completely bananas to you, but I think, like, things you build, things you create that you love and are passionate about, there's a bit of you in it. Like, there's a bit of your soul in the thing. So, like, you have imbued this thing with you, with yourself. And then someone's telling you to toss it out the window. You gotta be kidding me, you know? And so, so the, I think that when you're gonna change target and people have been working their ass off to build something meaningful to the old target, you just have to give it the morning cycle, you know? You have to be empathetic and human because you need them to come along with you on the ride.

AI assessment note: “you just have to give it the morning cycle, you know? You have to be empathetic”

Q Do you worry about becoming cynical with time?

A I kind of embrace becoming cynical with time. Because, I mean, we talked about one of my weaknesses is I'm too optimistic. Um, and I think that's unlocked incredible things in my life. Incredible things. But it gets back to if you haven't enrolled people in the vision, oftentimes you can't achieve the vision, and then it falls apart, and they're like, why were you such an idiot for being so optimistic? I'm like, well, if you believed we would have gotten there. And SMB. Suddenly you're going after the practitioner and the business person. You try to be all things to all people, and it's just way, way, way premature, and then you're nothing to anyone. You can choose the wrong narrow focus and implode, you know. So, that's the balance.

AI assessment note: “I kind of embrace becoming cynical with time. Because, I mean, we talked about”

Q Do you worry about becoming cynical with time?

A I kind of embrace becoming cynical with time. Because, I mean, we talked about one of my weaknesses is I'm too optimistic. Um, and I think that's unlocked incredible things in my life. Incredible things. But it gets back to if you haven't enrolled people in the vision, oftentimes you can't achieve the vision, and then it falls apart, and they're like, why were you such an idiot for being so optimistic? I'm like, well, if you believed we would have gotten there. And SMB. Suddenly you're going after the practitioner and the business person. You try to be all things to all people, and it's just way, way, way premature, and then you're nothing to anyone. You can choose the wrong narrow focus and implode, you know. So, that's the balance.

AI assessment note: “I kind of embrace becoming cynical with time.”

Q When you change your destination, or you realize that actually what you thought it would be, it's no longer, how do you communicate that effectively without being like a flimsy leader? Do you know what I mean? It's like, hey, I know I said that we were doing this, but like, I was wrong.

A It's so ingrained in being a human, you know? Yeah. So, and it ties to the thing about like, willing something into existence. Like, I think, and this, this will sound completely bananas to you, but I think, like, things you build, things you create that you love and are passionate about, there's a bit of you in it. Like, there's a bit of your soul in the thing. So, like, you have imbued this thing with you, with yourself. And then someone's telling you to toss it out the window. You gotta be kidding me, you know? And so, so the, I think that when you're gonna change target and people have been working their ass off to build something meaningful to the old target, you just have to give it the morning cycle, you know? You have to be empathetic and human because you need them to come along with you on the ride.

AI assessment note: “you just have to give it the morning cycle, you know? You have to be empathetic”

Q Product reviews that, you know, I'm not in product, obviously. They're kind of hailed as this kind of black box. How often Often do you do product reviews? Who's invited? Who sets the agenda? Can you just walk me through it?

A Well, like, ok. So, it's very different from, like, a brand new thing, a brand new feature, brand new product. In short, there's like, it's like fractal. So, I use a product all the time. I'm reviewing the product when I'm using it, like, continuously. And so, getting into, like, the preview, we have this dog food system and stuff like that. That's kind of like, uh, uh, you know, Asynchronous product review. But then, you know, in the classic scrum sprint methodology, you, you demo every Friday, you know. You want the core group to be seeing things as it's being built, because you can't waste time, and if it's going the wrong direction, you need to know soon, right? But then, you should have, like, continuous product reviews for the core team, and then, you know, monthly or quarterly, For management. But the danger, the danger is, like, the seagull management. Like, everybody-

AI assessment note: “continuous product reviews for the core team, and then, you know, monthly or quarterly”