Vals
other on 1 show · 5 statements across 1 episodes
5 statements about Vals, every show
Chi: Vals consumed $1.5M in model tokens in one month, 10x salaries
“In that month we spent roughly 1.5 million dollars worth of tokens. This is free, by the way. I, no, I don't want to but it was actually 10 X more we were spending in tokens than employee salary for that month.”
Chi: Vals AI committed to never sell training data to labs
“At VALS, one very early decision we made was the decision to never sell training data to labs. It's often a place that we're pushed. When we start working with a new lab to actually source and sell for them a bunch of training data.”
Chi: Legible enterprise evals will be AI adoption's biggest long-term bottleneck
“And I think long-term that will be actually the biggest bottleneck, our ability to take companies and their evals and make them legible because that's how we'll figure out what signal we hill climb on and where we actually adopt.”
Chi: Vals automates human evaluation work with internal system Steve
“We also have this internal system called Steve. Steve the Economic Vals employee. And so that, that's been a mechanism by which we're able to actually take more of the human work over time and put it into Steve.”
Chi: Vals runs massively distributed evals at maximum model rate limits
“Now we built up a team, but we've also really invested heavily in infrastructure. And so we're able to run evaluations in a massively distributed way running effectively the maximum possible rate limits with every model we get access to.”