The Exchanges

Q And, uh, fast forward through that early, um, phase. When did data become math?

A Yeah, so there's a chapter on data's mathematical baptism where data takes on the, the sacredness of the academy, and in particular, this scientific way of knowing things by applying mathematics to it. That chapter opens up with a hot IPO, if I remember correctly. The, the, that chapter opens up with the hottest IPO in the late 19th century, which was Guinness. So Guinness, the beer company, IPO'd in late 1800, and like literally people were breaking the doors down to try to get on that, get in on that IPO. Guinness had, like, all of the money, and so they could afford the hottest tech of the day, and they hired the hottest nerds of the day who were the statisticians, except they called them brewers. Brewers was, like, the great title, like, chief data scientist of the late-

AI assessment note: “hottest IPO in the late 19th century, which was Guinness”

Q Great. All right. So to keep it, um, going, hopefully that gives, uh, a flavor for this, uh, really interesting, uh, book that I've, uh, enjoyed reading, uh, how data happened. Uh, I'd love to use the last few minutes to, um, zoom out and talk about the New York Times. So again, you are the chief data scientist. What, what does data science mean at the New York Times?

A So I started as chief data scientist at the New York Times in 10 years ago. Actually, this is, this summer is my 10 year anniversary there. Um, so data science still means sort of a more orthodox definition of, of data science from 10 years ago, which is developing and deploying machine learning. So the data science team is about a 22 person team that develops and deploys machine learning. For newsroom and business problems. Most of the projects are things that, um, are relevant to many different companies, certainly to many subscribe, uh, subscription companies like machine learning that actually controls the paywall that decides when you should be asked to become a paying subscriber. Recommendation engines, which is not just personalization, but also identifying what's trending and then serving it in a variety of different surfaces. Uh, fancy ad products, so we can, you know, create advertising that's useful to marketers, but, um, is also privacy forward. Um, marketing, so when we market on other advertising platforms, that is done not using guessing and pointing and clicking, but using Python and optimization. Uh, that's a variety of things. We, we have a couple of things that are editor facing to help editors understand the relationship between stories and how they're promoted and how people engage with the stories. Uh, but a variety of problems that are all about developin…

AI assessment note: “data science still means sort of a more orthodox definition of, of data science”

Q To the extent you can, can you talk about, um, the tech stack for the data science team and data team in general at the New York Times? I don't know, data warehouse tools that one uses?

A Yeah, that's been a wonderful journey. Um, it was, so when I showed up at the New York Times in 2013, if you wanted to get your hands on data, you needed to write your own MapReduce jobs in Hive and hit buckets of unstructured JSON sitting in S three. Um, then we decided, sorry, then it was decided that we should build our own Hadoop on-premises, um, which was the style of the time. Then all of that went away, and, um, through a story that we don't have time to go into, uh, we started kicking the tires on GCP, Google Cloud Platform, and at this point we have fast, reliable SQL access via, um, Google, Google Cloud, um, and via BigQuery, which has made life so much less painful. That said, there is also a lot of work being done in AWS, and plenty of developer work happening on Amazon's Cloud, um, so the data stack is, in my team, the data stack is SQL and scikit, and occasionally Go, so it's scikit-learn is, is a particular module in Python where most of the machine learning you're going to want to do is already done, Um, a lot of containerization. Um, we still rely heavily on big table because a lot of big, big query, because a lot of times we want to score something and putting, put it in a table so that the analysts have fast, reliable SQL access, um, to the, to the output of those models. Um, I think that about it. Occasionally we code and go when things really need to be per…

AI assessment note: “in my team, the data stack is SQL and scikit, and occasionally Go”

Q asking is always the same thing as a bunch of people here who are looking to hire data scientists. So, you know, do, do, if I'm a startup CEO and I'm looking for my data scientist, am I, you know, do, do I focus on somebody who really understands, you know, ads or finance or health, um, or do I find somebody who is, you know, very sort of horizontal?

A Think about hiring Tukey or Leo Breiman or Some of the anointed great applied statistical scientists who worked as consultants. Their skill wasn't because they had great expertise in, you know, educational testing service, in the case of John Tukey. It's because they were great listeners, and they had respect for the people they were collaborating with. So, I wouldn't say that you need to go higher, I mean, if you look at Renaissance Technologies, right, they didn't win because they hired a bunch of MBAs, right, they hired a bunch of people who, you know, clearly were good at, at understanding how to represent that problem in terms of a quantitative field. So, I think what you're looking for is not a particularly somebody with a domain background, but somebody who's proven themselves to be a good listener.

AI assessment note: “what you're looking for is not a particularly somebody with a domain background”

Q What happens during World War II with code breaking? How is that important?

A So computers were born of a data science problem, which is a story that's not often told, and in fact was, was intentionally opaque in history for about 75 years. Moreover, I grew up as a physicist thinking that, you know, physics really won World War II, but now that I'm a data scientist, I realize that it was actually data science that won World War II, but that story was classified for about 75 years, which is the story of how the first digital programmable computers were created at Bletchley Park for dealing with streams of messy data. Any of you who deal with streams of messy real-world data will know that that's a huge pain in the ass. They had that pain in the ass in Bletchley Park, which was a Remote little place in England, sort of the Los Alamos of England, right? It's between Cambridge and Oxford, but you can't get there from here, and so you put something secret there. Um, anyways, so they had to invent special purpose digital hardware and electronic hardware in particular for solving the problem of dealing with streams of messy data. It's a story that's completely occult, and it was done entirely by people who were absolutely not statisticians, right? It was this mix of You know, puzzle programmers, mathematicians, and people who worked for the telecommunications industry in England That story had its own mirror on the other side of the Atlantic in Bell Labs, and h…

AI assessment note: “how Bell Labs played a crucial role in scaling up code breaking as a computational problem”

Q Meaning that you let the data tell the story as opposed to you trying to start with the hypothesis and verifying the hypothesis with the data?

A Um, a few years ago I would have said that, that that's what we do is we let the data speak and reveal unto us what are the really interesting covariates, except for two things that I think temper my, um, temper the extent to which it's that pure. One is, you know, there's a lot of computer, amateur computational social science that goes into figuring out how are you going to represent a person. You know, maybe you should represent the person by, ah, how many times they visited, or how many times they went to these different webpages, or maybe a feature that represents they've been to the website more recently than they used to, or they haven't been, you know, maybe they average every once a week, they come back, or something like that. Maybe you've got their zip code, and that itself is a join key on an abundance of data about that zip code, or something like that. But you, you do a lot of feature engineering in general in predictive analytics. So, I would say that there's a limit, I mean, I try to let the data speak for itself, and I try to do things that generally are in a hundred or a thousand dimensional, Things, so like I'll have hundred or thousand covariates that could be interesting, and then use the appropriate machine learning to reveal, ah, a small interpretable predictive model. That said, you know, we, we only, you can't discover a covariate that you didn't put in…

AI assessment note: “there's a limit, I mean, I try to let the data speak for itself”

Q bit to this point about, ah, being able to work with other people, that, that, do you find that the, or did you find that the people in the New York Times, maybe not the top people that made the decision, but the people that you actually need to work with on a daily basis, did, did you find that they were ready to embrace a, a data-driven approach culturally?

A Um, there was heterogeneity in sort of dataphilia and dataphobia. Uh, I would also say that the Times is an organization in amazing transition right now. Because of the transition that journalism is going through, not as a craft, but as a business. Um, so it's difficult to answer that question because it was, it's very different depending on to whom you speak in the organization, and I would have made no predictions about who would be dataphilic and dataphobic. I mean, there were some, there's some, there's some people who I work with who are, you know, old timers at the New York Times, meaning they've been there for like 20 years, who were very pro data, and are very pro data science. Um, so, it, it, it's sort of been embraced differently at all levels, but also, you, I, I like to use Steve Blank's definition of a startup, that a startup is a, is a temporary organization in search of a scalable and repeatable business model. And in that sense, every publisher is now a startup, because the business model of publishing just completely evaporated, 2004 through 2008. Like, Print advertising spend in the United States lost about 50% of its value in like four years, 2008 to 2012. Uh, and so, and that, and you've seen, you know, Chicago Tribune sold, Washington Post sold, Boston Globe bought by the New York Times and then sold. Like, there's been just tremendous change in newspapers …

AI assessment note: “there was heterogeneity in sort of dataphilia and dataphobia”

Q So to start from the beginning, the book starts at The end of the 18th century or beginning of the 19th century, and you make the point that data, the earliest form of data, was used by states. Um, tell us more about, about that. What did states do, and how did it all start?

A Yes. But no. The book doesn't actually start in the 19th century, right? The book starts in the classroom. So the book starts with me trying to convince the students that understanding data and how it got that way would be relevant for understanding their present day, and I'm trying to explain some mathematical concept, concept, and one of the students raises his hands and says, can we talk about Facebook now? And the reason he said, can we talk about Facebook now, is because five years ago, Mark Zuckerberg was in front of Congress testifying. In fact, and today, Sam Altman is testifying in front of Congress in exactly the same way. Society was concerned. Where did these algorithms come from? How did it get this way? And so five years ago, I was teaching this class and students wanted to understand, like, how do we relate? How do we understand how it got that way? How it came to pass that we've got these algorithms shaping our personal and political realities. And the claim of the book is that it's helpful to understand the present day, to look at that arc, And understand 200 years of people trying to make sense of the world and society through data. But to get back to Matt's question, yes, one of the things we try to do is to try to help people understand why these words are so confusing and malleable and used by different, ah, communities over different times. Why is it that …

AI assessment note: “But to get back to Matt's question, yes, one of the things we try to do”

Q Very good, and as a heads up, um, I'll, uh, open up to questions in, uh, you know, like a couple of minutes, see if anybody, um, has a question in mind. Um, LLMs, is that something that you guys at the New York Times think about, or use, or plan on using?

A I definitely have nothing publicly to disclose about any of that. I would say that the people in the data science team are generally aware, and actually not just in the data science team, the truth is, people who are at the New York Times read the papers, So, uh, I would say everybody I've talked to, not just the data science team, is well aware of LLMs and is thinking about What it means both for the New York Times and beyond, and there's obviously many different potential use cases, right? It's a technology, and we open up the book by quoting Kranzberg's first law, technology is neither good nor bad, nor is it neutral. A technology is a capable, a capability, and that capability is made mobile, and different people can use that capability for different things, so the New York Times has a variety of potential use cases for this. It's also been very useful because it's sort of turned everybody's attention to Artificial intelligence writ large and how machine learning more generally can be useful. So language models have been around for a while and natural language processing has been around for a lot longer. So it's also opened up to people the curiosity to think, oh, how could I use natural language processing in new ways? Generative, generative artificial intelligence as well as just natural language processing itself in my process. So it's opening up a lot of really interest…

AI assessment note: “I definitely have nothing publicly to disclose about any of that.”