The Exchanges

Q did you think about solving and maybe Ryan, you want to take this, that issue of the difference between kind of, this is what I was expecting versus this is correct. And the fact that maybe at a lot of companies, those two things were distinct or, or was that more like a three Oh one problem that you were working on? Not kind of a one Oh one problem.

A Yeah, it was definitely a one-on-one. Our v.one.o was a human loop solution, and so we built open source models. It would take a first pass. It would be, you know, generally correct. We would have people with internal tooling review all the output before we shipped it to our customers to view in the interface, and we originally hired Amazon Mechanical Turks, and we said, hey, we'll save money. We'll have three of them review every response. But we'd often see that even if all three of them gave the same output, we would go to an expert researcher or a customer or look at the data ourselves and often disagree. And then, so the AI might come up with one thing, our human, the loop process might have another, and our customer might have it. So it's actually three different inputs, and getting all three to agree, to agree was, you know, very challenging. And so we brought on a world-class researcher who was actually on the faculty at Yale as one of our other You know, very early team members. She was actually the fourth person to join SPRIG, and her first role was reviewing. This is a very experienced, you know, world-class researcher, you know, who is studying quantitative research at Yale. Come on and actually be that individual who is stamping and approving and tuning the AI. And so we at least knew that we have someone with some level of expertise And even though it's one person…

AI assessment note: “Yeah, it was definitely a one-on-one. Our v.one.o was a human loop solution”

Q Can you talk more Ryan about what your end to end product development process looks like now? And maybe in what ways would you say, would you say it's similar to the pre LLM world? And in what ways would you say when you're building a feature today, it might look different in some way.

A The uniqueness that we found for product development specific with AI is that You know, we define product best in class product development as a product manager and a minimum designer starting from the very beginning of ideation and building a product spec and a design prototype together. We'd love to have engineers involved sometimes due to, you know, time constraints and other goals are not always able to join. I think the unique part with AI is that you have to have An AI engineer involved from day one, because there's a question of the feasibility of what you're looking to build. And I think that's the big difference is that a best in class product development process has an engineer involved from day one. Uh, but with AI, it's actually going to be a requirement because there's been a lot of times where maybe our product man, you know, someone on our product management team and design team come up with something that they want to do with AI. And then they go to Kevin and they might spend You know, a month working on something, and we quickly find out it's not something that is feasible. And so we see more of a, you know, iterative process of testing with, here's an idea that we want to try. Is this something that is possible with the current model that we're working with, or maybe a different model? And then once we start to get the green light that, hey, there is some, you…

AI assessment note: “I think the unique part with AI is that you have to have An AI engineer”

Q And Ryan on the, on the sales side, how does that kind of vision setting fit into your sales process?

A We have a specific, you know, AI vision deck, and then if you go to sprig.com slash AI, you know, we've publicly shared our vision, uh, for AI. And so I think it's just going back to validating that with the current customers and then in the sales side, really seeing, you know, what is standing out and asking, you know, what parts of this are most interesting and applicable to the company that, you know, you're at and, and hearing that feedback. Um, but we're seeing a lot of companies Very similar to digital transformation. There's an AI transformation that's happening. A lot of them are saying we have been tasked with finding and applying AI to the work that we do. You know, this is an OKR. This is a company mandate, you know, from the CEO and taking what we're saying to them and pitching internally and saying, this is how we want to apply AI for our role, for how we develop products here at our company. And so that's where there's a lot of sales enablement of telling them this is how AI should be transforming how you develop products and really setting that narrative for them to take with them.

AI assessment note: “We have a specific, you know, AI vision deck”

Q Given, I assume kind of your take on the world is there's going to be all sorts of new, interesting technology coming out and you don't want to have dependency risk on a specific LLM, for example. And does it mean that you're approaching building products differently today than maybe previously? Because there's this desire to be able to use whatever the best is at that given point in time.

A One of the key inputs that, you know, I've been parting with Kevin on and, um, is really understanding the bounds of each model at any given time. You know, we often think around the jobs to be done that we want to solve for customers. We have to look at the different models that are available and see what can this model do for us today? And can we test the various use cases and applications for what we want to do with these various models that we've got? And it's very much Like a, you know, I have a one-year-old and I'm not going to ask her to ride a bike today. It's not a task that she would succeed at, you know, but I can't ask her to open and close her hand. And she's very successful at that. And I remember in the early days, a lot of people actually asked us to also ingest support tickets and other type of unstructured data into Sprig. And we knew the bounds of the model at the time were not successful with unstructured, uh, customer support Tickets. And so what we did was we relied on and encouraged customers to ask with a template gallery, very specific survey questions about specific moments in time and ask, for example, how was your experience using this feature for the first time? And you get somewhat like a narrow set of data back and the model at the time, it was very successful with a narrow set of structured data. And so I think that's a, an application, you know,…

AI assessment note: “really understanding the bounds of each model at any given time”

Q models, was there a period of time that you were worried that it could be a disruptive innovation for the company that in some ways the core of what Made the company special with this really hard work that you did in these ML features. And now it made it much easier for anybody else to do these types of things. And did that inform your strategy in any way?

A Certainly, certainly expected. It was a matter of when and not if knowing that AI at some point would allow the barrier to build the technology to be lower and lower over time. I think the key question that we just want to make sure we focus on is we're always ahead of the competition. In the field of AI. And I think that's the dimension that we take a hard look at and, you know, we are seeing other companies in our space add, for example, a survey or study level summarization. Uh, but while they're launching summarization of an entire survey, we're adding the ability to ask custom questions, you know, summarization and ask custom questions. And as they're starting to think about, you know, maybe whatever is next for them, we're moving on to, you know, product level insights. And something that some investors pushed back on in the early days of set at some point, this will not be a competitive image for you, but for us, we will always make sure that we are the most innovative company in the space. And we just lined up a really large contract, uh, with a very high profile company that, you know, everyone listening to this podcast has used at some point and they purchased because of the AI vision that they saw for what we're going to be working on. And so we just announced our AI vision as well. Uh, last month and what we want to do with AI. And so as long as we continue to push …

AI assessment note: “Certainly, certainly expected. It was a matter of when and not if”

Q What else can you share about what you're hearing from customers as this kind of tidal wave has sort of been built up? Do you see sort of a level of enthusiasm and excitement in the abstract that just It feels very different than conversations you were having, call it, 12 months ago. And are there sort of strategy implement implications of that in any way?

A I think we're seeing a level of excitement from customers, but also they have to be okay delegating some of the control they previously had. It's that balance between man or woman and machine. And human and machine. And with the new study level AI that we're shipping right now, it summarizes the entire study, you know, with a summarized output. And you can ask it questions. How should I improve my product experience? And I think a lot of people are starting to realize that they need to be comfortable actually delegating something that they previously felt like was a decision that they had to make and be comfortable with moving higher up in the order of decision-making and see AI as a true co-pilot, where I think a lot of people still are trying to retain that control of all the decisions or do all the analysis, or maybe make their own decisions on what they want to do, you know, with this data and how can we actually move, you know, our, the users using Sprig and I think the broader, uh, individual working with AI to focus on higher level orders of work. And I think ultimately as like a species, you know, we're now so much further along if we can trust And actually find ways to work with AI that allow us to leverage our time more effectively. And so I think that balance, we're seeing a lot of people kind of uncomfortable with that balance and seeing about how they fit in and wo…

AI assessment note: “we're seeing a level of excitement from customers, but also they have to be okay delegating”