The Exchanges

Q What's in your AI stack along those lines?

A The PMs are mostly using vZero. The designers love Figma, so they're using Figma Make. Uh, the engineers are using a, a combination of tools right now. Um, so Cursor, Cloud Code, GitHub Copilot. Marketing teams use all sorts of tools for translation, subtitles, you know, content adaptations, et cetera. Customer support uses Intercom Fin. So there's quite a lot of tools that are kind of used across the company. I would say though, that something that is Kind of annoying to me is that we haven't yet figured out The bridging from the tinkering to the workflow quite as seamlessly as I would like. Right. And so each sub function, even though the common, I guess, wisdom now is that AI is going to strip away this, these like functional titles. It is kind of true that based on your experience, like you may gravitate to using a type of tool more. And if that tool isn't as interoperable with some of the other tools that you need to pass down the chain to actually ship it into production, at least at our scale. Right. I think for smaller startups, sure. PM should just go ship it. But for us, like we are still doing some handoffs between functions. I expect that to change over time. And we are investing in some of like, you know, design system components and MCPs and stuff to make it a little bit easier. But yeah, it's a, it's an investment and it takes time to, to smooth things out.

AI assessment note: “The PMs are mostly using vZero. The designers love Figma, so they're using Figma Make.”

Q Speaking of that, you mentioned this with Duolingo is just very good at habit formation and motivation behavior. Feels like chess is good at this too. You've worked at both these companies. What have you learned about how to motivate people, how to create habits?

A Again, like Duolingo would not have started without this, uh, insight from day one, right? They aim to, to focus on motivation and build a lot of these like tactics. Um, Jorge actually had this model of like gamification, uh, patterns having essentially three pillars to it. You have the core loop. You have the, uh, metagame and then you have the profile. And so we actually thought about it that way too, where, you know, your core loop is, is your lesson that you go through. You do a lesson, you get some rewards, you extend your streak, and then the next day you get a push notification. It's kind of the core loop of the product and making that really tight is super important because people need a habit to stick to. Then you need a metagame, which for Duolingo is kind of like the path, but it's also the leaderboard achievements, kind of long-term things that you're going to strive to such that you have like long-term I guess, motivation, uh, to continue doing the thing. And then the profile is also critical because you build up a profile over time. It's reflection of your investment inside the product experience. And so when you nail those three things, you can end up with a long-term learning journey that can be quite successful. And then to flip over to the chess.com side, like what we see is that over 75% of our new users, they classify themselves as like, I'm completely new t…

AI assessment note: “having essentially three pillars to it. You have the core loop. You have the metagame”

Q Magnus is, is what you said about 2800. Yeah. And then the stock fish is, would you say 3630?

A Yeah. And, and really it's like, it's because computing power is so amazing and there's so many techniques for how to do like deep evaluation on specific chess lines. They can calculate tens of millions per second. So it's not realistic for, for a human to compete against that. But yet, like watching some of these chess engines played has opened up a lot of creativity, new strategies, new lines, new appreciation for the game. And our chess.com approach is that we can bring this technology For every user, even people that have never moved a piece before. I talked earlier about that game review product. That's exactly what this does. So behind the scenes, we're running chess engines to, to basically spit out evaluations for every move that you make. And then we translate that and make that approachable to the user using, you know, their native language and plain approachable, uh, style.

AI assessment note: “Yeah. And, and really it's like, it's because computing power is so amazing”

Q Okay. Very cool. Okay. Uh, is there a product you've recently discovered that you really love?

A Yeah. So last 20 years of my life, roughly, uh, I've moved around a lot, but I've always been within walking distance of a coffee shop. It's just like a ritual that I go and get coffee and it starts my day. Right. Two years ago, I bought a house and for the first time ever in my life, I'm like not by a coffee shop. And I was so depressed about this for a little while. So my favorite product is, uh, is the bread bowl, uh, barista and it just starts my day off. Right. Um, I like making horrible latte art with it, and, uh, I think it's just a reminder, I don't know. Like, the products that most Um, impact me, I guess, are the ones that, uh, I use all the time, and it's a daily habit, just like my, and then the most caffeine. You got it.

AI assessment note: “my favorite product is, uh, is the bread bowl, uh, barista”

Q Following that thread a little bit more, people listening to this are imagining, how do I get better at experimentation? How do I run more experiments? How do we do this better? What are two or three tips and best practices that you think people need to hear or maybe are not totally aware of when they think about getting better at experimentation on their teams?

A I think the first thing is just start somewhere, you know. I, I just read this Alassian Uh, state of product report. And it was like, 40% of product teams, like basically don't run experimentation at all. And there may be some good reasons for it. I mean, it could be philosophical or maybe you're more, you know, B to B oriented or whatever. So I, I get it. But I think for a lot of, especially if you work on a consumer product that has some degree of scale, some degree of frequency with your product, you can collect enough data. And also I have found, you know, I can pattern match all day long. I've worked a lot of companies, right? But I'm wrong all the time. And I think consumer behavior can be very fickle. And especially when you work at a company, you become a power user naturally. So sometimes you, you may forget like what the actual user experience is for a brand new user. And so you leave a lot of opportunities on the table if you don't even try to experiment. So I just encourage taking that first step, just run an AB test, find a third party tool or something that you can integrate quickly and, or, or even just work with your engineers to spin something up, just get in the practice of, you know, Crawled and walked and run type of thing.

AI assessment note: “I think the first thing is just start somewhere”

Q a bunch of recent guests, which is many people were very good at piano when they were younger and were very serious piano players. For example, head of ChatGPT, Nick Torelli was like almost going to become professional jazz pianist. You were very serious in as a piano player earlier in your career. How did you go from pianist to one of the top growth minds in the world briefly?

A Well, that's very flattering, um, but I appreciate it. Yeah, I, I grew up playing a lot of piano. Um, my parents were, were immigrants from Taiwan, and I was the oldest kid that they had, and so I definitely felt that strong, uh, encouragement, if you will, to learn a bunch of things, take them seriously, study hard, and so I did, right? And like, my parents, even though they weren't musically proficient, they had a, like, deep love for classical music. So I was a stereotypical, like, baby that would listen to Mozart, I guess, when I was sleeping type of thing. And I still vividly remember, like, we had this upright Yamaha piano, and at the very top of the piano, we had this countdown clock from 90 minutes, literally every single day of my childhood, um, just practice really, really consistently. At first, like, I really was irritated by that thing, but as I grew older, I started to appreciate, like, music quite a bit more. But anyway, like, I think what really accelerated my, my interest and abilities in, uh, piano was like, I, I feel like I hit the lottery. I had perfect pitch. And so I was able to, you know, quickly understand whether I was like playing the right stuff or the wrong stuff and just pick up music pretty, pretty rapidly.

AI assessment note: “I grew up playing a lot of piano. Um, my parents were, were immigrants”