small models

“I would say if we are aiming for AGI, there is a token crisis, but if we are aiming for useful small models, I don't think there is a token crisis.”

“I think the things that people see now is like the small models don't really handle nuance as well, and they could be more repetitive if, even if they have really good instruction tuning, but if you take that kind of seven to seventy billion parameter jump, li…”

“So I think text extraction is like one use case where small models can be really performant, and it makes sense to use them instead of just using larger models.”

“If you set these models up to use tools, They just won't. Like if you say, hey, here's a question. You have access to these tools. Do as many rounds of tool calling as you want, and then submit your answer. They'll just submit their answer because they like ar…”

“I think that we should establish the definition of small model as being a model that a grad student can inference at reasonable time on a single GPU. Which is probably like seven B maybe. I don't think 27 is small under any reasonable.”