The Exchanges

Q you were, you were showing something that people all need to see, is it's these kids in the inner city who were failing and suddenly succeeding. They weren't, clearly weren't failing because they were dumb or because they were incapable. It was, it was clear they weren't interested or didn't care or didn't think it mattered to get good craze, right? I thought that was a really interesting insight, right?

A The incentives weren't there for them, right? And it was interesting, one of the, an eleven-year-old really schooled me one day in New York City, uh, and I think it was in Crown Heights. And he, I asked him, well, why do you like the experiment? And he says, oh, I said, is it the money? He says, no, it's not about the money, man. He said, although it's nice. He said, in every other aspect of life, I'm, I'm basically need to be an adult. But when I come to school, I'm not treated like one. And so for him, the incentives, this really was his job. Now he could focus on it. And I never had, I mean, you know, sometimes you, you do the right thing for the wrong, for, for lucky reasons. And I never thought of that as a mechanism, but these kids really liked the idea of making this, what they were truly focused on in their occupation. We do that in middle-class and upper middle-class households all the time, but in the inner cities, not so much. And so I was, I think that was one of the reasons it worked well.

AI assessment note: “The incentives weren't there for them, right? And it was interesting”

Q education work you did, which I love, uh, the most noteworthy probably was your study on police use of force, right? And this was in 2016, shortly after the shooting of Michael Brown. Uh, I apologize, you probably told the story a lot of times, but can you reprise it briefly for us? What did you discover about the police use of force, and why did it piss people off?

A Yeah. Well, you know, look, I was upset because I was watching, just like you and everybody else, these videos come out, and we were watching this, and, and everyone had to be thinking, is this really the norm in America? You know, and we, we watched and we watched cities in protest over these videos and I wanted to do something about it. Now, as you know, I'm an economist, economist, and I believe in comparative advantage. So protesting is not my comparative advantage, but I'm a data nerd. So I decided, let me see, let me go collect some data and I will demonstrate that police are biased. And then, you know, that'll help. That'll help this movement along. And so I collected millions and millions of data points, uh, both on the lower level uses of force, putting your hands on a suspect, throwing them up against a car, maybe hitting them with a baton and other physical types of force that are not lethal. And then we also collected a lot of data on actual lethal uses of force, uh, officer involved shootings, for example. And what we found was that on the lower level uses of force, there are racial differences that we couldn't explain. Uh, and we put them through formal bias test, and there is bias that goes on in the lower level uses of force. What we didn't find was any, that that bias actually, uh, Was still there as the force ratcheted up. So for officer involved shootings, we…

AI assessment note: “for officer involved shootings, we found no racial differences and no bias”

Q meritocracy. Roland believes we could actually use data to solve a lot of these problems rather than just sit here and yell and talk about it. Let's actually figure out what the data says, and let's apply that and make the world work better for everyone. Let's chat with Roland about it. I think in a really challenging situation in a broken family, you know, how'd you make it out?

A I don't know how I made it out. I, I had, um, a couple things maybe, Joe. I, I, I had a lot of people who took an interest in me. My grandmother was a school teacher and was, you know, phenomenal, uh, even though, uh, she wasn't my parent. She liked to say she brought me home from the hospital, and she did. Uh, you know, I had teachers along the way, but also, and I hope this doesn't sound too, uh, self-absorbed, but I was a big dreamer, man. You know, I, I, I had things I wanted to accomplish in life, and there were people in the neighborhood I grew up in who were like, you know, to show how tough you are, you should go to prison. And I was like, that doesn't make any sense to me, right? Like, why would I want to, I got customer service needs. Why would I want to do that? Um, and so I, I really, uh, thought I was going to be good at something, and I was willing to put in the work to, to, to do it. And it started out as, as a, as a little toy athlete, you know, playing football at age five years old, but that gave me the confidence to know if you really put in the effort, you can be great at something. And so throughout life, when things came up, you know, my father went to prison, et cetera, or people told me I wasn't going to amount to much. I just knew that, um, Probably because I was a dreamer that, that, that big things were possible if I put in enough effort.

AI assessment note: “I had a lot of people who took an interest in me.”

Q that and disparate impact in general, because it seems like this disparate impact thing has stopped us from having a lot of these merit-based tests and tests in general. So to a lot of us, government just got stupider over the last 40 years because you can't even Quiz people anymore. And yet people seem to be still obsessed with disparate impact. Like, are they correct? Like, why is that?

A Yeah, it's not how I would think about it. I would say, look, it's not whether or not there's a disparity in the test scores that, that should cause you to throw out the test. Although, they've been doing this, as you say, forever. They continue to do it. I remember years ago that on the SAT, they threw out the following test because it was biased. Seven is an integer. Could you pick out the other, uh, integers? And they threw that out because they had disparate impact against, um, Non-minoraries. So it's just, it's just a culling of test scores to make us all look like, to blunt the instrument. Here's what I would do. I would say, look, um, is the test equally predictive for the groups? Yeah. That's what matters. And if not, then maybe you want to use different instruments for the different groups, right? I know that's controversial, but the idea is what you're trying to do in your mercy, in the most earnest way is figure out what predicts success, right? That's what these instruments are for. And so if you told me that, look, there were really, really skilled black people who scored very low on this test. In aggregate, at scale, not two people, but on, on average that that was true, that it was essentially zero correlation between the test and a objective measure of productivity for black people, but for white people, it was different. Then I would say, well, this test is not…

AI assessment note: “it's not whether or not there's a disparity in the test scores”

Q Is there any way to give like really short tests or certain types of questions that help you with the cohort analysis or help you learn more? Cause people might be asked their resume. Like how do you. Yeah.

A Yeah. And that goes into the phase two, right? Phase one I described earlier is we take the data you already have and build models where it gets really fun is what you just described when. You know, if you were operating a franchise, you and I, Joe, would come in and say, all right, like, what should we ask at the time of hire? What can we experiment with that's going to have the highest predictability? Could be a short test, as you described. Um, it could be that we ask, you know, um, all sorts of, of other types of open-ended questions and allow people and allow AI to actually analyze the answers. Really, we, and that's where, The fine-tuning of the models to particular companies becomes a lot of fun. That's how you help people win is because the executives have real hypotheses about the types of, uh, uh, characteristics and traits that are really successful for them. What I'm trying to get away from is these kind of corporate terms like, you know, I want, uh, You know, radical teammates. Like, what does that mean?

AI assessment note: “Could be a short test, as you described. Um, it could be that we ask”

Q But it seems like a lot of things went away from that. And you've seen a movement in tech and, you know, a lot of these companies to now embrace merit again and use that to help everyone. You know, I want to ask what, what are some of the results to date? Like, what are some of the examples? Who are you working with? What can you tell us?

A Well, I don't know if I can, I don't know if I can name names here, but I think that, um, Tell us about some wins. We are, we are working with, uh, Corporations that range from in size from 230 employees to, you know, 700,000, um, uh, global franchises. And, and it's all the same, um, type of problem, which is particularly in food and Bev or in, in, in, uh, hospitality or, um, other entry-level jobs. It's all about figuring out how to Uh, reduce attrition by hiring the right people, right? And most of the products out there, Joe, are all about saying, let's just hire faster. And, you know, less friction in hiring is always a good thing, but what if we could hire faster and With higher predictive quality. And, and that's what we're helping companies do. So it's, it's, um, we're, I like nothing more than helping innovative CEOs win, right? And, and the way you win is that you get the right people on the door. And this is not, this is not, you know, early 2019 nineties HR I'm talking about. This is using the most sophisticated analytics on the planet to be able to finally Um, breakthrough in an area that I think has been a real pain point for, for a lot of executives, which is how do I use data in a, in a serious and sophisticated way to, um, to increase the, the, the talent available to me to reach my goals.

AI assessment note: “I don't know if I can name names here, but I think that”