language models

“From next week, we're going to build our language models in the open and share what works and what doesn't work.”

“The next step is one to one. It remembers what you're asking for, like a cookie or an embedding. And then it's you plus a thousand of these language models all going and doing your bidding, the agent based kind of thing.”

“They were designed to be reasoning machines, not fact machines.”

“So I think in some cases it is a feature. If you want to use a language model for creative writing and if you want it to be really, really creative, then you probably want it to hallucinate. So in a way it's a spectrum of groundedness and hallucination where i…”

“At this particular point in time, probably yes, but just because the field is moving so incredibly quickly. And so I think that in the next year, we're going to see lots of other models coming out. And if you can have a language model, agnostic AI company that…”

“If things did not change, the language models, or the models in general, would all converge towards, they're all training on the same data, and at some point it's what mix of data you use, but you'll trend towards the same answer.”

“What has happened for language models is coming for computer vision, for images. The democratization, democratization, how do you simplify it? Then there's the intersection of those true multimodal languages, which there are many of them out there, but how do …”

“I think that we are seeing the flattening off of the value of just adding more compute and more data to language models.”

“That happened, like, in 2017, shortly after we submitted the paper. We started training language models on Wikipedia and we sampled from those models, and it could write Wikipedia pages as convincing as a human page.”

“I think it's a good idea for countries to have infrastructure within their countries. Like, I think it's a good idea for people to have power plants in the country. You know, I like that Canada has several nuclear power plants and has several water power plant…”

“So I think the idea of saying like, oh yeah, you got to learn how to prompt is going to go away. I think the idea of saying you need to learn how language models work and you need to know what they can and can't do in the same way you had to learn how a comput…”