AI product development strictly requires an AI engineer from day one
Ryan Glasgow · A guide to building product in a post-LLM world | Ryan Glasgow and Kevin Mandich from Sprig · Sep 7, 2023 · at 37:39
Ryan Glasgow is the founder and CEO of Sprig, an AI-powered user insights platform. He discusses how product development methodologies differ when building AI-first features versus traditional software.
0:00 / 0:37exact quote · 37.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The uniqueness that we found for product development specific with AI is that You know, we define product best in class product development as a product manager and a minimum designer starting from the very beginning of ideation and building a product spec and a design prototype together. We'd love to have engineers involved sometimes due to, you know, time constraints and other goals are not always able to join. I think the unique part with AI is that you have to have An AI engineer involved from day one, because there's a question of the feasibility of what you're looking to build.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Ryan Glasgow
Defining correctness in AI outputs is becoming increasingly subjective
“As we, I think, trend towards artificial general intelligence, the definition of correct or the definition of whether something is, you know, valid or not, It's becoming increasingly subjective”
Much of AI companies' value now lies in prompt engineering
“And I think a lot of the value for AI companies now, you know, is in prompt engineering. I think the early days was actually just Can you build a model? Can you get the model running? Can you get the model working in production? Where I think a lot of it is, c…”
ChatGPT succeeded partly by remaining a standalone product outside core workflows
“And when you look at ChatGPT, I think partly why it's been successful is that it's such a standalone product that's not integrated into anything else. And I think if it actually was integrated into your Slack or into your email or into your work, I think peopl…”
Agentic AI workflows are technically feasible today but face reliability hurdles
“Based on what we've seen today in the boundaries of the LLMs today is that we could, based on what we have tested with the models, actually build that now. And we are seeing the sophistication, the capability of the models to build that now. I think the roadbl…”
Fast AI evolution requires hiring for learning slope over specific domain experience
“Making sure that the slope is there is really critical as someone who's really smart, just because the AI field is changing and particularly this year, every week, it's rapidly changing. And so you're going to need someone who you can't rely on their prior exp…”
Three-worker Mechanical Turk consensus often failed expert data accuracy checks
“We originally hired Amazon Mechanical Turks, and we said, hey, we'll save money. We'll have three of them review every response. But we'd often see that even if all three of them gave the same output, we would go to an expert researcher or a customer or look a…”