Structured Outputs

“We are not retrying, you know, we're doing it in one shot and this is how you save on latency and cost.”

“The schemas, we support recursive schemas, and this allows you to do really cool stuff, like, you know, every UI is a nested tree that has children, and so I thought that was super cool. You can use one schema and generate, like, tons of UIs.”

“Like if something is 95% reliable, but you're chaining together a bunch of calls, if you magnify that error rate, it makes your like application not work. So that's a really exciting thing here from going from like 95% to a hundred percent. I'm very biased wor…”

“All our models support it, or all of our newer models support it, but we don't support it with structured outputs right now.”

“The model before was able to hallucinate a tool, but now it's it can't when you're using structured outputs.”

“Yeah, we didn't use any kind of Other stuff. We kind of built, you know, our solution from scratch to meet our specific needs.”

“A lot of people were hacking function calling to get the response format they needed, and so this is why we shipped kind of this new response format. So you can get exactly what you want, and you get kind of more of the models verbosity.”

“We trained a model which is significantly better than our past models at following formats. And we did the end work to serve like this constrained decoding concept at scale.”