Test Time

“Assigning higher costs to tokens that come at test time, because once you're at test time, it kind of implies that a user is waiting. Something is waiting. It's either another process, a user, an event, and there is real cost to like every single token or like…”

“Most of the evaluations and work in this area, they kind of assume you get all of your context at test time. You get a math problem, you get like the entire setup and the question at test time.”