Insight
Defining correctness in AI outputs is becoming increasingly subjective
“As we, I think, trend towards artificial general intelligence, the definition of correct or the definition of whether something is, you know, valid or not, It's becoming increasingly subjective”
Insight
Much of AI companies' value now lies in prompt engineering
“And I think a lot of the value for AI companies now, you know, is in prompt engineering. I think the early days was actually just Can you build a model? Can you get the model running? Can you get the model working in production? Where I think a lot of it is, c…”
Opinion
ChatGPT succeeded partly by remaining a standalone product outside core workflows
“And when you look at ChatGPT, I think partly why it's been successful is that it's such a standalone product that's not integrated into anything else. And I think if it actually was integrated into your Slack or into your email or into your work, I think peopl…”
Opinion
Agentic AI workflows are technically feasible today but face reliability hurdles
“Based on what we've seen today in the boundaries of the LLMs today is that we could, based on what we have tested with the models, actually build that now. And we are seeing the sophistication, the capability of the models to build that now. I think the roadbl…”
Insight
Fast AI evolution requires hiring for learning slope over specific domain experience
“Making sure that the slope is there is really critical as someone who's really smart, just because the AI field is changing and particularly this year, every week, it's rapidly changing. And so you're going to need someone who you can't rely on their prior exp…”
Disclosure
Three-worker Mechanical Turk consensus often failed expert data accuracy checks
“We originally hired Amazon Mechanical Turks, and we said, hey, we'll save money. We'll have three of them review every response. But we'd often see that even if all three of them gave the same output, we would go to an expert researcher or a customer or look a…”
Prediction Not checkable as stated
AI models will likely never reach total accuracy for subjective feedback categorization
“Cause we will never, it's very unlikely we'll ever get to a hundred percent. And so we'll get, you know, 90, 95, 99. But we'll always make sure that they have the ability to then do that last mile of analysis on their own. And so that's something that we have …”
Assertion Not checkable as stated
Many AI startups failed between 2018 and 2020 over human-in-the-loop dependencies
“And a lot of startups actually went under in 2018, 2020, just realizing that they were not going to get there.”
Insight
A startup's current AI model may be surpassed within three to twelve months
“And so I think for any startup, given the advancements in the LLM field, the current model you're using might be the right model today, but three months from now, 12 months from now, there's a good chance that another model might actually surpass the current m…”
Insight
AI product development strictly requires an AI engineer from day one
“The uniqueness that we found for product development specific with AI is that You know, we define product best in class product development as a product manager and a minimum designer starting from the very beginning of ideation and building a product spec and…”
Disclosure
Pitching a product as an automated researcher alienated early customers
“I remember at first the very early days of Sprig, there was some wording around your automated user researcher, you know, or it does all the text analysis for you. And that was actually very off putting for both user researchers and product teams.”
Insight
AI product teams must present concrete options rather than asking what to build
“You know, I think with working with customers with AI, you also have to develop world-class techniques of not asking the customer what to build, but instead giving them a set of options to provide feedback on.”
Insight
AI engineers must be embedded directly in product squads, not isolated
“And so having, AI engineers embedded with traditional engineers has been the big breakthrough for us and working together To develop and build these features and not seeing AI as, like Kevin said, an input output, you know, the model spits out a response that …”
Prediction Not checkable as stated
Many companies will hire AI architects to sit alongside the CTO
“Going forward many companies will bring on AI architects that might even be at the same level or report directly to a CTO to think about what are the models that we're going to use and what are the boundaries of AI and how can we build reproducible AI and thin…”
Insight
AI startups lacking workflow or data ownership face imminent disruption
“If you are, you know, a founder or someone working on AI, you really need to think about if you're not owning that core workflow, if you're not owning the data yourself, then it's very easy to be disrupted and someone else could easily, you know, and most like…”
Disclosure
Sprig hired a Yale faculty member specifically to review model outputs
“We brought on a world-class researcher who was actually on the faculty at Yale as one of our other You know, very early team members. She was actually the fourth person to join SPRIG, and her first role was reviewing.”
Ryan Glasgow