Agarwal: AI teams rarely use human feedback, despite high satisfaction when used
Rohit Agarwal · Why every AI Engineer needs an AI Gateway (ft Portkey.ai CEO) · Feb 5, 2025 · at 21:59
Rohit Agarwal, CEO of Portkey.ai, discusses LLM observability and human feedback loops with host Swix.
0:00 / 0:11exact quote · 11.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I think it's not used as much. I agree. And I think that's something I tell every customer that you need to close the loop and feedback is going to help you. I think the teams that are doing it are really happy. They're not doing it.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Rohit Agarwal
Agarwal: 90% of production LLM use cases do not use automatic routing
“In fact, I would say in production, 90% of the use cases do not use automatic routing. What they want is deterministic flows. As long as the gateway manages authentication authorization for them, it's perfectly fine. The request hitting A specific model that t…”
Agarwal: Gateway-based guardrails reduce latency versus application-level checks
“And again, you can decrease latency massively by building this on the gateway. Rather than building this within your application.”
Agarwal: Gateways produce the best LLM metrics without manual codebase instrumentation
“We're saying that the best LLM metrics can be produced directly on the gateway. Without every team having to instrument their own code bases to pick out these metrics.”
Agarwal: AI telemetry cannot standardize while underlying APIs change monthly
“Standards usually evolve when there's some amount of coherence and stability in an API. I think the APIs themselves change every month. So I'm not sure how there's going to be standardization that occurs.”