The Wisdom Wall

“You still want to do a PhD if you're an entrepreneur, or at least a masters, because you want to really sort of learn deep. I mean, you might be doing this by yourself. You don't have to, but it's useful because you, you learn more about, you know, what exists out there, what's possible, what's not possible, what, uh,…”

“The most likely business model, uh, that has to do with AI is taking a open source foundation model, like LAMA, which is the data open source system, which is used everywhere now, right? Every, almost every startup uses it, uh, even large companies. Uh, so take an open source platform, um, whether it's an LLM or an,…”

“For almost every problem that we have, the cause is really a lack of knowledge or intelligence by humans. We're making mistakes. We're making mistakes because we're not smart enough to figure out we have a problem, because we're not smart enough to figure out solutions, we're not smart enough to organize ourselves to…”

“It's very inefficient because the system has to try many things before it gets the correct answer. Um, and so, It's very inefficient. It requires many, many, many trials. And so it works really well for games. You know, you, you, it's very efficient. If you want to train a system to play chess or go or things like…”

“And the reason it works for text and not for other things is because text is discrete. So there is a finite number of possible things that can happen, right? There's a finite number of words in the dictionary. There's a, you know, so if you can discretize your signal, then you can use those autoregressive prediction…”

“LLMs are system one. The architecture I'm describing, which I call objective driven AI, is system two.”

“scientists, you try to understand the world. Um, engineers, you try to create new things, and very often, if you want to understand the world, you need to create new things. The progress of science very much is linked with progress in technology that allows to Collect data.”

“we, we, we use that term, it's an abuse of language, because those neurons are not really neurons like in the brain. They are, they are to real neurons as an airplane wing is to a bird wing, ok?”

“So the combination of those three things, you know, having already a number of skills that you know, experience with solving problems and accomplishing tasks, being able to learn new tasks really quickly with a few trials, um, and then the next step is being able to solve new problems zero shot without having to learn…”

“chatbots are, uh, or LLMs, large language models, are a special case of that, where you train a system to predict a word, But you only allow it to look at the words, the words that precede it, you know, that are to the left of it.”