Agarwal: AI gateways manage material per-request costs unlike traditional API gateways
Rohit Agarwal · Which AI Model are companies actually Paying For in 2026? | Rohit Agarwal, Portkey · Mar 19, 2026 · at 1:06:31
Rohit Agarwal, founder of AI gateway Portkey, explains why managing LLM API traffic requires specialized infrastructure compared to legacy database routing.
0:00 / 0:35exact quote · 35.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Every API request is literally that has a dollar value to it. Like an Opus request with 10,000 tokens is probably 23 cents, and this is, you're making a thousand requests, which means this is 23 dollars worth of API requests that you've made today, or two 30 dollars, whatever, that map. That is what makes this gateway a lot more valuable, because it's not an API request that is so minimal in value that it doesn't even matter. Like a GET request to the DB request, you don't even think of it as cost. Now this is like, it's 23 cents, it's real, and I'm showing you it's 23 cents.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Rohit Agarwal
Agarwal: AI operational spend will surpass traditional cloud DevOps spend
“So will AI spend surpass cloud spend? I a hundred percent believe that, which means that the AI ops spend surpasses DevOps spend. It's bound to happen.”
Agarwal: Claims that 80% of code is AI-written are accurate
“There are so many companies coming out and saying, 80% of all of my code base today is written by AI, and that is truth, and that's what we're seeing everywhere as well.”
Agarwal: Anthropic's Claude Code is killing the standalone vibe coding startup market
“It is killing that market. I think there is, that market is now finding its own niches, and Anthropic clearly realized that if 50% of my API revenue is going to come from vibe coding, they will Claude code and be like, I can still build the best vibe coding sy…”
Agarwal: AI startups must prove they offer better returns than backing Anthropic
“And I've heard this in conversations where a VC would be like, do you, do I invest in startups, which is like a risky business and I'm still getting a 10 X return or do I just put my money in Anthropic and I'll just get a 10 X return without doing any work. So…”
Agarwal: SaaS will become systems of record while AI agents handle execution
“In the current format, SAS companies will just become systems of records. So you had the database, which got obsolete because of a SAS platform, which is just crud on this database, but interesting crud on this database. Now, SaaS platforms I think are the new…”