Barry McArdle: Context building and model chaining are AI's hardest work
Barry McArdle · Embedded AI: The Questions Every CEO is Asking · Jun 2, 2023 · at 25:41
Barry McArdle, co-founder and CEO of Hex, explains that integrating AI capabilities requires complex context assembly and prompt engineering rather than simple API calls.
0:00 / 0:30exact quote · 30.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And I think that is really actually the hard work in many ways of Incorporating AI into your product, right? Like it feels really simple. Like you can go get an API. Anyone listening to this today could go to the open air website and get an API key. It's everything you're doing before you're sending requests to that API of like building the right context, understanding how you're iterating on how different models are going to respond to different types of prompts and different amounts of context, how you're potentially even chaining together different types of models or different modalities of models together in one sequence in UX.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Barry McArdle
Barry McArdle: AI foundation models and API calls are already commodities
“I think the models themselves and being able to send a request to one of them is, is already a commodity.”
Barry McArdle: Hex does not use customer code to train AI models
“We're not using it to train models. I should be very clear. Like that's one thing we've been very upfront with our customers about, like we're not training models where they would expect to ever have like some code they've written be a completion for someone e…”
Barry McArdle: Hex enforces a rule to never automatically run AI code
“We have a rule as we're building these features. It's like, we don't run the code for you. Like we will generate the code. We will show it to you, but we want to keep the human in the loop.”
Barry McArdle: Passing too much context into LLM prompts confuses the model
“I think we've been, we were tempted early on to try to shove as much context as we could in. You kind of figure like the more I can tell this model, the better. And you realize you can pretty easily confuse a model in terms of the amount of context you're pass…”