Raycast

“Basically what we came up with, it's essentially you just write a TypeScript function and you document your TypeScript function with JSDoc. And then we extract all the information from there and basically make that and pass that information to the LLMs.”

“So we basically came up with a system that they can write like a DSL for those evals, and then they can run the evals themselves.”

“And we're not using, like, a proxy, like a router or something like this, so we built that internally.”

“And so we looked into all the various models we had and then we picked, at the moment, it's gbd-for-o and gbd-for-o-mini, Which we basically did a fine tune to really optimize for our use case, and then basically shipping that in the app as Ray one and Ray one…”

“So all of those AI extensions going to be open source. We have 50 at the beginning.”

“Which OpenAI would be figured out at some point is, like, oh, there is an upper limit of, like, a 128 Tools you can have. And then the API basically gives you an error. So we ran into some of those issues as well.”

“And so we find like the open AI models function calling wise for our use case. So for the best ones and like, yeah, basically verifying the others. We had a beta group and testing different models. So basically from all providers and yeah, find basically the b…”

“Nobody talks about how to do evals with tool calls. It's, like, a super unsolved thing.”

“So this is primarily like using like OAuth and then doing API calls and then developers basically can expose those information to our AI and then we're picking it up and composing those information together.”

“So tomorrow we're going to release our next big feature, which is called AI extensions. And that basically brings together our roots of like having developers extending Raycost. And this time they can extend Raycost AI.”

“Like our future is basically turning Mac OS and we also now expanding the windows and iOS and really like a AI native operating system.”