LLMs

“When it came to the massive leap forward in LLMs recently, we staffed a team to really prototype quickly and discover what was possible and just apply hypotheses outside of the typical norms of how we work. So they kind of went straight to prototyping instead …”

“The promise of the UI has to match the quality of that underlying data. Which is to say, and I think that's actually one of the failings of the various LMs right now is they all appear supremely confident, even when they're completely hallucinating.”

“Likely LLMs will improve, like we'll probably be able to create better and better blue. So all sorts of software systems can talk to each other. And so again, very broad idea, but yeah, I think we'll see a lot of very successful companies where that's the kern…”

“Because the way a link before LLMs and the link, you know, 10 years ago was actually an HTML link. And now Google can read content so they can say, well, you've been mentioned here. That's pretty good. So now we acknowledge that this might be the brand and thi…”

“You need to make sure that your product gets better as the LLMs get better. And that too many people are building things to make up and compensate for the LLMs, that all that work is going to go away. So it's okay to do it to understand that it's going to go a…”

“So much of how the industry is built presupposes the idea that software is expensive to write and cheap to run. And LLMs undermine both of these.”

“One of the reasons I'm excited about LLMs being a disruptive technology is I think that it allows us to get out of this monolithic sense of like, whatever, we're all just beholden to a decreasing number of very powerful organizations and lead into Everybody be…”

“Mostly I find that like the straight up LLMs themselves are good enough.”

“At Conform, we get a ton of inbound customer requests, as you can imagine. Coming from the field or directly from customers, we use LLMs to take in those asks to summarize what they're about, to find other asks that are like that one, like really in a compelli…”

“LLMs cannot predict deals because it's like very, very specialized.”

“How this is going to work, if you fast forward, you know, one, two, five years, whatever, like PMs are, they're going to be writing quote unquote, instead of just writing a Jira ticket and waiting for a developer to do it. The developers are going to be able t…”

“Software is deterministic. When you write code and you hit run, either runs or it doesn't. And that is why that's the key insight Anthropic really had. They just went deep. And this is what they're doing. It's just reinforcement learning. I'm basically permuta…”

“These LLMs are not great depending on how big your application is though, right? These things are not quite there where if you have something that's, you know a thousand files or something or more Where you're going to be able to have a really reliable, super …”

“If you can scope down when things get really big, if you can give it a smaller task to work on a specific component or a specific file, you decrease that likelihood of the LM not being able to reason over or very, very, very long context windows.”

“There's going to be a fundamental engineering skill that's going to be useful for decades or centuries to come, which is creating foundational infrastructure. Think about LLMs in the, in terms of, they're like oracles that can go and write software for you. Bu…”

“I will admit that I have tried this and I think it's a great idea. I think it does help. I have not gotten to a point where I'm like comfortable sharing it with other people nor have I told anyone that I have tried to do this for them, but I've certainly done …”

“There's one controversial take for me working with LLMs is don't use TypeScript. Use plain JavaScript. Use JSX. It's easier for models to write code this way.”

“As LLMs get better, it's like people like 10 X engineers would have way more impact. They were going to be a hundred X engineers because they're able to manage LLMs. And it's not necessarily the team size, no, the funding, ah, that, that would be able to win y…”

“As the nature of an integration, I think it's potentially slightly less strong than building everything full stack built in. And once we did that, and actually I engineered the endpoints and the SDKs and whatever to work well with LLMs. I think Base-Forty-Fort…”

“One phenomenon we've seen when teams are building things really quickly with AI is that the more AI generated or assisted they are, the more generic they tend to turn out, which makes sense if you think about how LLMs were developed, you know, they're all basi…”

“So that idea of working static software isn't true anymore. It's almost like living, breathing software and ensuring that the things that allow it to breathe actually occur.”

“You have LLMs, which have generalized pretty, you know, like in fascinating ways, but they've kind of also asymptoted in that you know, we have run out of data for the most part.”

“Usually when you ask something like what's the best tool for X, the first answer will be mentioned at the most in the citations. So that's very different from Google.”

“The more you answer all the questions, the better. If you don't answer a question, then you're probably not going to show up, and if you answer a question, this follow-up question and subtopic that somebody else is not answering, you're gonna be more likely to…”

“We are seeing a fair amount of variance in answers just without doing anything at all. So you definitely want a control group.”

“Dot dash Meredith is a large media conglomerate with good housekeeping, all recipes, Investopedia. It's the, it's probably the most successful SEO company of all time. And it's also one of the most cited, probably the most cited in LMs as well.”

“LLMs know what open codes are, and they know what axial codes are, because it is a concept that's been around for a really long time. So those words help me shortcut, like, what I'm trying to do.”

“What's new here is that you can't figure out your rubrics upfront. People's opinions of good and bad change as they review more outputs. They think of failure modes only after seeing 10 outputs they would never have dreamed of in the first place.”

“You know, interestingly, LLMs themselves are quite bad at playing chess. Like they hallucinate moves. They look at patterns, right? They're very good at pattern recognition, but not so good at going super, super, super deep on a specific chess thing.”

“So behind the scenes, we're running chess engines to basically spit out evaluations for every move that you make. And then we translate that and make that approachable to the user using, you know, their native language and plain approachable style. And even wi…”

“LLMs are non-deterministic, right? Now we can't just Put in a command and get something back and accept it. We really need to evaluate it. So, you know, are we seeing hallucinations? What's the reliability? Does it meet like the style that we would typically w…”

“I'm in a camp where I do believe that something new will be needed.”

“Agents fundamentally, like LLMs, can't tell the difference between texts that you give them and texts that you copy and paste in from other people. They're all the same thing. So instructions in that input text can always override the earlier instructions.”

“These LLMs and even video models, they don't know how to do that. They don't have the ability to understand friction or weight or contact pressure friction, surface texture. Like they're just not able to do these things. And this is the core of what we need. A…”