LLMs

“A very popular use of these LLMs is to generate synthetic data to train other models, because they're really good at generating sort of long tail data.”

“I think the past month has demonstrated that that is not true, that, you know, a lot of existing companies can actually act pretty swiftly to integrate LLMs into their applications.”

“And I think that that is the promise of LLMs and startups. It's not to build better versions of things that exist today, but rather to kind of reimagine some of the tools and systems that we use in an LLM centric way.”

“So I wouldn't go all in on this unless you can have like a human in the loop who's like helping.”

“If we're building this new age of L-empowered applications, The kind of requirements for the type of data that like you want to load as well as how you want to extract information from that data will be a little bit different than the existing ETL stack. The r…”

“I think that LLMs in particular, we talk to a lot of the people and we don't see a ton of people getting them into production yet. And I think it's funny, like VCs are always surprised, like when we tell them that I think that I don't know. I'm bullish on LMS,…”

“I think all of the major LLMs out there, almost all were trained using weights and biases.”

“In the LL world, like the anecdote is something people really pay attention to.”

“We see that this whole RAC architecture is probably the first one that will really see a broad, massive adoption, right? Simply because it's all about accessing and, you know, somehow working with information that is already around. And this is where we see, l…”

“The only meaningful way to adopt to adopt NLP and LLMs is in the cloud”

“LLMs are, I believe, the new IP.”

“We're going to do the calculations because the LLMs are very poor at that.”

“One of the drawbacks of using LLMs is like, we're not receiving confidence scores. On the extractions.”

“You actually need to have a gateway between your application and your LLMs in order to track all of the costs and understand before moving things to production how much you can anticipate in terms of cost.”

“Actually, for me, LLMs happened or had a similar momentum, like, two and a half years ago, where, like, we started also building and seeing infrastructure being built around training and deploying language models, and then we expect the same thing to happen wi…”

“We don't have any equivalent set of theories for intelligence or artificial intelligence. We have a lot of theories of how some bits of it might work. But we do not have a theory of what we have and what, and in what sense is what we have is different and the …”

“We can't do that with LLMs either. We don't know what will happen if you put double the data in or why, or we don't know why it works with this much data or not.”

“A lot of those companies to put it bluntly don't have a lot of traction yet, because it turns out that, if you're trying to do something, and I can, I'm not picking on them, but like evaluation or monitoring and all the things, well, you need to have LLMs to m…”

“One of the challenges that will happen with some of these large language models, particularly when they're chained or we let them operate for hours at a time is the intern effect where they're off at the beginning. And that's just because they're chaotic and n…”

“We discovered that many of our customers actually use multiple LLMs.”

“I don't think, and I actually don't, in the book, I don't think just scaling up the current LLMs will get us to, I don't, I'm not even sure they'll get us to AGI, let alone super intelligence.”

“These models, when they're general they're optimizing for what's known as generalization error, or the average error across all examples it sees on the internet. And as a result, it's pretty good at everything, but it's perfect at nothing.”

“A couple of years ago, you know, a lot of companies try to train their own LMs. Like that to me, it makes almost no sense, right?”

“Those things are stochastic, not deterministic, meaning that they don't get the right answer a hundred percent of the time, and you don't get the same answer each time you ask. Therefore, evaluating performance is more important arguably, than any other kind o…”

“By the way, editing files with models is actually quite a tough problem. Turns out they don't know how to generate diffs. They don't work with line numbers very, very easily. So you have, you need a quite a complicated system for edits to work.”

“What we found is reasoning over long contacts is actually not very good. It's like, not good at all. Like, once you cross 32,000 tokens, that performance of reasoning, you know, just goes down to hell, like, very, very quickly.”

“The script generation part we don't take on because we think the LLMs and stuff will solve this eventually. I think they're still not very good at video scripts, and they're getting better and better, but we do think that as these models improve, this will be …”

“Because I think of all the possible sort of AI unlocks, like even compared to LLMs and stuff, there's probably nothing more dangerous than video. Generation in a way, right?”

“You know, it fails randomly two out of 10 times, which might be fine in a supervised setting like ChatGPT. You know, where you're talking with these sort of assistants but it doesn't really work in an automation case where it's hands-off keyboard running on a …”

“This is why Google and Meta didn't launch their own LLMs in twenty-twenty-two when they had them as well, because they looked at them and said, well, they're wrong too much.”

“As you get more ambitious with the things that you wanted to do, like create more interactive code, data fetching code, et cetera, you always end up with an agentic architecture because you realize that LLMs are not bulletproof.”

“By almost any definition anyone could have written down, let's say five years ago or 10 years ago, certainly when, you know, Tim, you and I started our PhD. We basically have the vision of AGI that, that we had back then. We have things that can write code. Th…”

“I know for a fact that big companies are training now LLMs in-house. Really, like, big companies who have the financial means to train chat to be like model are hiring people who train LLMs.”

“For blog writing on book writing, not so much because honestly, I, for fun, I tried it out. It's just, it generates okay text, but it's, I don't know, it does not I can ask it to generate text like me, but it's almost like, then I don't like it and I end up ed…”

“Because if you are Meta or Google, You've got this whole other highly profitable business, which now needs to have LLMs inside it, powering all sorts of capabilities and features, and you probably want them to be your LLMs rather than somebody else's. But you …”