Knoop: LLM failure rates break unsupervised server-based automation workflows
Mike Knoop · Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop · Apr 3, 2025 · at 33:01
Mike Knoop, co-founder of Zapier and NDEA, shares customer feedback on why language models struggle when applied to hands-off automation.
0:00 / 0:15exact quote · 15.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“You know, it fails randomly two out of 10 times, which might be fine in a supervised setting like ChatGPT. You know, where you're talking with these sort of assistants but it doesn't really work in an automation case where it's hands-off keyboard running on a server, right, where you're not, like, actively supervising it.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →