Unmanaged LLM API Calls Cause Generative AI Costs to Skyrocket, Brodbeck Warns
Marty Brodbeck · How Generative AI Is Changing Travel — With Priceline's Marty Brodbeck · Dec 12, 2023 · at 10:59
Marty Brodbeck, CTO of Priceline, discusses the operational expenses and caching strategies required for running generative AI in production.
0:00 / 0:15exact quote · 15.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Cost in, in running and managing a generative AI platform could skyrocket. So you have to be very smart about when you refresh or make direct calls to the large language model and this third party information.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Marty Brodbeck
Brodbeck: AI Personalization Depends on Underlying Data Architecture, Not LLMs
“Well, I think this is more about how companies design and develop their data architectures than generative AI understanding personalization on the individual level.”
Priceline Generates AI Marketing Content for Top 1,000 Global Destinations
“So we've taken the top 1000 city destinations in the world and have been systematically creating content in those three different channels I talked about.”
Priceline Saw Mixed Results in Generative AI Code Automation Pilot
“So we've just recently completed a pilot of a product that played in the generative AI space as it pertains to code automation. And quite frankly, we had mixed results.”
Generative AI's True Developer Value Is Automating Maintenance Tasks, Says Brodbeck
“And that's where I see the real value in generative AI is unlocking a lot of the maintenance and support functions that developers have to do and having them solely focus on Building great features for your product.”
Static Audience Tiers Fail Because Customer Travel Contexts Constantly Change
“What we're really striving for is really one-to-one personalized, a one-to-one personalized product experience, because The same person that you could put into an audience the next time they come back to Priceline could be looking for something completely diff…”
Priceline Rolls Out Generative AI Checkout Bot to 100% of Users
“We ran tests to make sure that the bot had a positive impact on the user experience and the, you know, and revenue, and after several iterations of prompt engineering, we got to a point where the bot was A-B testing positive, and so we pushed it out to a hundr…”