Gateway

“People really want the problems to be solved, like sort of somewhere else, like in the model or in the gateway, but more and more we see it's the harness that solves these problems.”

“The only way to do that logically is to find a way to aggregate agent traffic somewhere. If it goes through a certain gateway, a firewall, or some router, I can watch all the traffic and I can stop an agent from acting. That's the only way it works. So I said,…”

“Yeah, I would say 70, 80%.”

“So later on, they say, okay, in that case, it's we'll pay you two percent of all the revenue you help us generate on TV. And on top of that, we'll pay you 20,000 dollar a month.”

“So the gateway, when it runs, has a memory footprint of like, 21 MB, and the way we've been able to do it is just using a very interesting architecture where we're using transformers in JSON to do everything out of the box.”

“We're saying that the best LLM metrics can be produced directly on the gateway. Without every team having to instrument their own code bases to pick out these metrics.”

“You actually need to have a gateway between your application and your LLMs in order to track all of the costs and understand before moving things to production how much you can anticipate in terms of cost.”

“When AMD launched the first one gigahertz processor, we couldn't get our hands on it, so Nelsonerotics came up with the idea, just buy a bunch of gateways that have the processor in it. We'll take out the one gigahertz We'll put a cheaper two or 300 dollar pro…”