The Exchanges

Q And in terms of channels, so you're particularly prominent on LinkedIn. Uh, again, if I'm a startup, uh, doing technical data science stuff or AI stuff, uh, would you recommend I start with, with one channel and just focus all my energy on that, or do all the things that you mentioned, so like YouTube and podcast, or like, how do I, how do I start?

A I would repurpose content as much as possible to make it as easy on yourself as you possibly can. So if you're already writing blog posts, Why not convert that into some sort of newsletter that you can put on LinkedIn and on Substack, right? You're literally just copying and pasting stuff, and you're kind of trying to reach a couple different new people, but you think about it as just like top of funnel inbound. You'll deal with it later and see who comes through those channels once you get a little bit more established, right? You just kind of trying to start someplace. Um, You already, you have a newsletter, right? So cut it up into some smaller pieces for some LinkedIn posts as well, right? Like take some paragraphs, take some high level learnings, ask people some questions. LinkedIn is a great place to get to know your audience as well. People are really involved in the comments and leave thoughtful comments from my experience. So that's where you can start to ask people about what they're excited about, what part of this resonated with them. You'll notice who responds to what kind of content, where on your LinkedIn. Now you've got that kind of going, let's say you found some LinkedIn posts that worked really well. Well, why not hop on a video and read it and read through some of the comments and discuss what you learned. Now you can edit that into short form clips for Inst…

AI assessment note: “I would repurpose content as much as possible to make it as easy on yourself”

Q gaming world in terms of, um, you know, the role of data science, like the way data science organizations, uh, are, um, put together in those companies. Is that, is, is that a reasonably new thing that they would be data scientists in the, in the first place? Like, how do they, how do they find their way in the world of, of gaming, which historically has been pretty insular?

A Yeah, that's a great question. I'd say that it's not necessarily new, like within the last couple of years, um, organizationally taking data science seriously, elevating it within the org, you know, relying on like center of excellence models, I'd say is newer to gaming, right? I think everyone, all industries are kind of figuring this out as they go. I would put gaming right up there with everyone else, like trying to decide, Where in the org should your data team live? There's actually a decent number of choices, and when you think about, like, a publishing studio, um, not necessarily Activision, but any of the big companies that have a couple different gaming studios within them, right? Like, you have different owners of different pieces of different projects that you're working on, so, like, do you centralize your data team, and they kind of help everyone, Certain teams are going to want embedded data scientists because they have a big need for it, and what they're doing is, like, you know, iterative enough that they really need that expertise, and you need to be close to your subject matter experts, and there's a constant trade-off between those two things. Like, as soon as you centralize something, you will inevitably lose the deep expertise you can get from embedding, but as soon as you embed everyone, you lose that, like, you know, Center of excellence where everyone co…

AI assessment note: “I'd say that it's not necessarily new... organizationally taking data science seriously... is newer”

Q lessons learned or any recommendations you have for, let's say I'm a, you know, I'm a stale startup, uh, raised as, Sit around and, um, you know, my entire team is super technically strong, but like marketing is just not what I've personally gravitated, uh, you know, towards for a long time. So I need the help. So what, what, what do you recommend for people in that, um, case?

A I'd say that I think if we can all learn anything about what's been going on the last few years, really with like social media and the way people use products to me, and I'm probably biased, Your community is your most important asset, and so what you really need, you can have the best people, like you said, you can have the best product, but existing in obscurity isn't going to get you anywhere. Um, You need someone, or you can do it yourself, who you trust to build a community around your product and a community of people who trust you and who will evangelize for you. And without that community and that support, it's going to be really, really, really hard to cut through the noise because there's so many people with interesting ideas nowadays, and attention is a commodity just like anything else. And you're gonna have to fight for it, and you're gonna have to pay for it, and you're gonna have to be strategic to get it, and acting like That's somehow selling out or it's focusing on the wrong thing. Uh, I know cause I'm also a super technical person. Sometimes you can over index on the technical and getting maybe, you know, your platform to be five percent more fast or supported on one more browser might not 10 X you the way that spending that time building a community would. Because ultimately without that, it doesn't matter what browsers are supported. It doesn't matter how g…

AI assessment note: “build a community around your product and a community of people who trust you”

Q Do you think there's going to be a more kind of like on the fly personalization of like journeys within games?

A I would love that. There used to be, I'm blanking on it. There was a game a really long time ago that tried to do something similar. It obviously wasn't like generative AI in the way we're thinking, but it was like, A very, like, in-depth, like, choose your own adventure kind of situation. Um, I would love to see that, because again, you know, you're talking to something in the world, and, and you know that there's, like, two paths laid ahead of you. Like, at any point, you can almost see, like, the decision tree of where you're, you're able to go. It would be fun to kind of obfuscate that a little bit away from the player, and have them feel a little bit, even more immersed.

AI assessment note: “I would love that.”

Q I mean, great. Very interesting. Okay. Uh, maybe it's switching texts a little bit, um, and talking about, uh, rebel data science. So did you want to maybe explain what that is, what the firm does?

A Oh yeah, for sure. So that's my consulting company. The name came out of actually one of my followers, uh, at one point when I was really early in my journey was like, you should follow Carly. She's a rebel data scientist. And I was like, wow, has there ever been a more active Description of my personality. I would have gone with like chaos, data science, maybe, but rebel fits as well. Um, and so, yeah, I started this as a consulting company basically because I was getting a lot of questions from either people looking for help in their careers, but mostly brands who were looking for help with their data science, branding, how to be taken more seriously as thought leaders in the space. They were looking to partner with me on projects, you know, where we could amplify each other's like vision or, you know, give away something that they were excited about. Um, and I realized I needed kind of some umbrella to encompass all of these things that I was getting asked to do. Uh, and that's kind of where rebel data science came from. Um, I have a friend working with me on it now, so we are a team of two, a company of two whole people. It's an exciting time.

AI assessment note: “that's my consulting company... brands who were looking for help with their data science”

Q Yeah. So let's dig into this as super interesting. Um, I guess what are some of those pitfalls and conversely, what, what has worked? I mean, it, it sounds like a lot of it and, you know, my words, not yours, but like a lot of it seems to revolve around content and quality of content. So maybe let's, let's double click on that.

A Yeah, I think let's think about some pitfalls that I've definitely seen, um, misaligning. Your messaging with where you're at or with where you want to be, right? So who are you talking to and why are you talking to them? Having like very clearly defined audiences that you want to reach, right? Um, let's say for me, you come to me and you want me to talk on LinkedIn about like, you know, your, I don't know, data science product that sells plants. I don't know how we do that, but it doesn't. Um, And what you tell me is you really want to reach, uh, decision makers who are going to be building office buildings because they buy a lot of plants. You don't care about the individual consumer right now, but you don't ask me any questions about my audience. You're not trying to dig into what my messaging or my strategy would be. Maybe you're not even actually explaining to me your funnel. Maybe you don't understand your funnel at all, right? Lacking all of that context, you're never going to get out of social media, what you put in because You're either going to have too broad of a message that people are kind of going to be like, eh, or you're going to have way too hyper specific of a message with misalignment with your audience, and people are also going to be like, eh. Um, and that's where, like, the understanding of your community is going to come in to play extremely, like, it's e…

AI assessment note: “let's think about some pitfalls that I've definitely seen, misaligning Your messaging”

Q this a little bit, um, your journey into the world of gaming. I think you, you mentioned that you, you, you, that felt like you were, you found you people, but like, uh, again, the gaming industry is a, is, is, is fascinating, but it's historically been kind of insular, um, and kind of quirky. Uh, what, what was your experience sort of discovering that industry as a data scientist?

A It's interesting because I'd say on the data science side, um, from what I've seen, we have a lot more, uh, representation in terms of, like, different genders and people from different backgrounds, and I think it's because data science has historically been something that A lot of people like me who before data science was a thing could get into. So we were already used to being like, you know, one of my colleagues is a philosopher. I'm a chemist. We have physicists, right? Like it's just, there's no standard for that. It's changing a little bit, but it's traditionally been that way. Um, and I just, I see more representation in, in data teams than I do for something, let's say like the, I don't know, hardware level programming. You know, which has just historically been, like, a lot of these, like, deep nitty-gritty computer science fields have been, like, more skewed towards men, um, and so I'd say it's, the representation there is better, but I think that gaming as an industry as a whole, like, you can look at the data yourself, it still suffers from this idea that only boys play video games, and so only boys make video games, and, you know, it's really changing, I think, but it's, They're like these big ships, you can't turn them around instantly, you know.

AI assessment note: “gaming as an industry as a whole, like, you can look at the data yourself”

Q And lessons learned around observability. Um, you know, what, what, what matters most is that the, like the data lineage part is that the, you know, the detection is that the fixing is like, is that the reporting of it or all of the above?

A I'd say all of the above the what I've seen have the most impact, though, and where I see data scientists, science projects fail a lot is actually observability into like drift and label drift. So after the fact, I think a lot of when we started building data science projects, we were like, you have to be end to end, which is fine, you know, prototype to production. But like what production means and what that maintenance means is something that's actually kind of difficult Because how often do you go back? Like how much time do you have in a day to go check the distribution of your labels and check for drift? Um, a lot of people are working in this space, which I think is great, but it's just one more thing that you don't have to worry about. If you can get an alert, you know, if it seems like the distribution is changing, if your labels or something weird is happening, um, I think that could save a lot of projects from going off the rails once they're kind of in broad and you kind of don't know what they're doing and you don't have time to check.

AI assessment note: “I'd say all of the above the what I've seen have the most impact, though”

Q Yep. And, um, I think I heard you somewhere, uh, because I think that was part of one of the several jobs you had, um, at, at Activision, that there was a security component as well, like a, uh, intersection of, um, AI and security. What is that, uh, application or use case?

A Oh, let's speak more generally so that I don't get in trouble for my trade secrets. Um, Stepping back from gaming, because there's a lot of similarities, the way I like to describe this is think about you work at a bank, right? How would you go about detecting fraud on someone's credit card? Um, you might start with, all right, let's build up a history of their purchases. Like, what kind of stuff does this person usually do? You know, if it's me, they're buying video games. They're going to GameStop. Like, so if I end up buying, I don't know, like a handbag in Greece one day, you might say, that's weird. What's weird about it? Well, the Item is strange. The location is strange. There's a lot of weirdness going on here, right? And so this kind of idea of anomaly detection is pretty well known when you think about spaces like banking. I'd say that there's similar overlap in gaming as well, right? Because you're just looking at behaviors at scale over time, trying to figure out what seems normal, what seems abnormal. And how can you build in safeguards for abnormal behavior? Because not every abnormal behavior is bad, but abnormal behaviors are something you should look at. Um, the way I describe this kind of machine learning is there's a, there's a tenant of machine learning where like you just throw out the outliers because they're going to mess up your distribution and you kind…

AI assessment note: “let's speak more generally so that I don't get in trouble for my trade secrets”

Q in particular in the world of gaming, but not necessarily, uh, emerging use cases? So, you know, you mentioned graphics, uh, and then you mentioned the specific use case of security, but like, what, what, what do people in, uh, at a, at a place like, uh, Call of Duty slash Activision do with General AI? Do they experiment with it, play with it, uh, whatever you can talk about?

A Yeah, I'd say more broadly at, like, what I've seen publicly discussed at other gaming companies, um, especially at conferences like the Game Developers Conference, I can probably talk about more, um, because that's all obviously been disclosed. I sat in on a presentation of, I forget who it was, maybe it was ZeniMax, only to remember. Um, they were doing basically, Trying to figure out if they could use generative AI to insert the player's name into the game, right? And so I see actually a lot of this because people love open world games. But there's a joke on social media right now, right, that, like, everyone's an NPC, because they're, like, these unbelievable two-dimensional, you know, characters that don't really have a personality, and they don't really interact with the world in a way that makes sense. Um, and so there's a huge push to make NPCs not NPCs, if you get what I mean, right? And I think that this is a really cool use case, because the stakes to me are a little bit lower. Like, it's no one's, Full-time job to make an NPC, like say your name. It's just a functionality we've never had. So you're not really replacing anything people were doing. You're just making something that didn't exist exist. And it's kind of cool. So they were actually, you know, you're, you'd put in your name at the beginning, you know, you'd say, like, I'm, like, Spaghetti John, or somethi…

AI assessment note: “I'd say more broadly at, like, what I've seen publicly discussed at other gaming companies”

Q Having your data already, uh, where is your threshold? And I guess what, what are the, you know, the way you recommend people get started is that, uh, you know, you need to have like all, uh, modern data stack in place or go buy Snowflake or just put a bunch of, um, data on, on an S three. Like how do you, what do you typically advise?

A I think it's somewhere in between there. Like, let's say you have enough data that you're kind of You know, if you're not super technical, you're noodling around in spreadsheets, but it's too much for you to handle by yourself. You know, you're kind of like, ah, we can't really do what we're trying to do. We can't scale at what we're looking for. Um, but we do have the data available to, if I were to hand this to someone today, I'd be like, can you build something? And they could get it done in a week, right? If part of your equation is, um, We also have to make the data available. It's a fundamentally different conversation, and it's one that you perhaps should hire both people at the same time though, right? Because the ingester of the data and the producer of the data need to be on the same page. You don't have to do that. Um, but if you have the option to, I think that that's a, a decent way to move forward.

AI assessment note: “I think it's somewhere in between there. Like, let's say you have enough data”

Q uh, interestingly because, presumably because, um, the AI and machine learning is such a strategic topic and, and part of, um, what the company considers to be A really, um, important part of their, you know, IP and strategy in general. Uh, but to the extent you can talk about it, like, what does, uh, machine learning mean in the context of, of a game like, uh, Call of Duty?

A Yeah, that's a great question. So I think back, you know, even five years ago to some of the gaming conferences I would go to, and you think about the things that people were discussing, right? It's a lot of the same usual suspects, you know, graphics is a huge area of research. Um, but you started to see more and more this idea of machine learning applied to gaming specific problems. And now I feel like I can't look at any gaming conference without seeing Something that's like deep in the weeds for gaming. You know, maybe it's like synchronizing sound to like movement in the world, right? Like some interesting rendering slash audio problem, but there's a machine learning component of whatever they're talking about. So it's like, how did we 10 X our workflow with machine learning? Or how did we, um, Unlimit our artists with machine learning or generative AI, right? And so it's a lot of rapid iterations on making all of the people who were already doing amazing things so much more effective. And it's just been insane to see the explosion in this. Like, I feel like, you know, now you can't turn around with seeing another application of AI and machine learning in the gaming space. And it makes sense, right? Because what we do ultimately is very deep in Research, like I said, right, like computer graphics, obviously, like, right, who were the biggest purchasers of GPUs way back whe…

AI assessment note: “now I feel like I can't look at any gaming conference without seeing Something”