guardrails

“But you are able to put in place guardrails better if you yourself have used the technology.”

“When I talk about guardrails, the first thing that comes to mind is, oh yeah, we have to do PII redaction and sensitive data redaction. But somehow we've seen that the more common use cases for guardrails end up being, you know, is it the right length? Is it u…”

“And again, you can decrease latency massively by building this on the gateway. Rather than building this within your application.”

“In reality, we've seen what gets deployed the most is regex-based guardrails to say, you know, I want to catch for specific words, and then take certain actions based on that. Or I want to catch for empty outputs.”

“I do think they're finding clever and clever ways to lock down particular areas sometimes, but I think it's at the expense of capability and creativity. So there's some model providers that aren't prioritizing this and they seem to do better on benchmarks for …”