Prediction Not checkable as stated
Shazeer: Most Core AI Intelligence Will Come From Text Models
“But I think that a lot of the core intelligence is going to come from a From these text models.”
Insight
Shazeer Calls Next-Token Prediction an AI-Complete Problem
“And like, the problem is super simple to define. It's just like predict the next word, the fat cat sat on the, you know, like, okay, well, you know, what comes next? Like it's extremely easy to define. And if you can do a great job of it, like, you know, then …”
Prediction Not checkable as stated
Core AI Intelligence Will Primarily Emerge From Text Models
“But I think that a lot of the core intelligence is going to come from these text models.”
Prediction Not checkable as stated
AI Scaling Has Not Hit a Wall and Will Keep Improving
“So, and at the same time, like, I don't think anyone's seen a wall in terms of how good this stuff is. So I think it will just, it's just gonna keep getting better. I don't know what stops it.”
Prediction Not checkable as stated
AI Models Can Generate Synthetic Data to Avoid Looming Shortages
“I think I'm not that worried about Coming up with data and I feel like we could probably like just generate some more with the AI.”
Insight
AI Assistants With a Single Neutral Corporate Persona End Up Boring
“Like if you're like such a public, you know, trying to present like one public persona that everyone likes, you're going to end up just being boring essentially. And people just don't want boring. You know, people want like the, you know want to interact with …”
Insight
Deep Learning Succeeded Because It Matches Modern Chip Hardware
“Really the key insight is that what makes deep learning work is that it is really well suited to modern hardware where, you know, you have the current generation of chips that are great at Matrix multiplies and, you know, other, other forms of things that requ…”
Prediction Not checkable as stated
Researchers Will Eventually Solve the AI Hallucination Problem
“Yeah, I think, yeah, there's like a ton of great work going on in you know, in trying to figure out what's real and what's hallucinated, of course. I think we'll solve this.”
Assertion Not checkable as stated
Daniel de Freitas Built Google's Meena as a Scrappy 20% Project
“Like he started the thing as like a 20% project where like people are encouraged to spend 20% of their time, like doing whatever they want. So and then he just, like, recruited, like, an army of, like, 20% helpers who were, like, ignoring their day jobs and, l…”
Insight
Providing Emotional Connection Does Not Require High AI Intelligence
“Probably you don't need that high end, like level of intelligence, like
To, you know, to do emotion.”
Disclosure
Character.AI Enforces Filters Blocking Pornography, Violence, and Self-Harm
“Like, you know, we don't want to encourage people to, like, harm, you know, hurt themselves or hurt other people or you know, we're blocking porn.”
Insight
Shazeer: Text Is a Thousand Times More Information-Dense Than Images
“An image is worth a thousand words, but it's like a million pixels. So like the text is like a thousand times as dense.”
Insight
Self-Attention Brought GPU Parallelism to Sequence Modeling
“The insight here was, Hey, you can use the same attention thing to like, look back at the past of the sequence that you're trying to produce. And you know, the beauty is that the, that It runs great on on GPUs and CPUs, and it's kind of parallel to, like, how …”
Insight
Shazeer Estimates Text Is 1,000 Times More Information-Dense Than Images
“An image is worth a thousand words, but it's like a million pixels. So like the text is like a thousand times as dense.”
Insight
AI Data Needs Scale With the Square Root of Compute
“The data requirements tend to go up like with the square root of the amount of computation, because you're going to train a bigger model and then you're going to throw more data at it.”
Assertion Supported
Google's LaMDA Was Originally Just a Renamed Meena Chatbot
“So that was like a renaming of Mina. So I guess I went and helped Daniel on Mina. We got it on some giant language models and then kind of became like an internal, like a viral sensation and then got renamed to Lambda.”
Opinion
Big Tech Avoids Releasing Open-Ended Chatbots Due to Asymmetric Risk
“I mean, I think just like large companies have concerns around like launching products that can say anything. I mean, it's probably just a matter of like I w I would guess it's just like a matter of how much you're risking versus how much you have to gain from…”
Insight
Conversational AI Platforms Should Let Users Define Their Own Use Cases
“This is kind of a technology that that's so accessible that like billions of people can just invent use cases, you know, and like, it's so flexible that, you know, you really just want to put the user in control because often they know way better than you do w…”
Disclosure
Many People Turn to Character.AI for Companionship When Lonely
“We also see, like, a lot of
You know, people using it cause they're, you know, they're lonely or troubled and need someone to talk to.”
Assertion Not checkable as stated
Active Character.AI Users Spend About Two Hours Per Day on Average
“Somebody who's on the site today is active for about two hours on average today. That's of people who send a message today, which is pretty wild.”
Insight
AGI-First Companies Must Make Product Quality Depend Entirely on AI
“The way you have a company that's both AGI first and product first is that you make the, make your product depend entirely on the quality of the AI. Like the most, the biggest determining factor in the quality of our product is, is how smart things gonna be.”
Assertion Not checkable as stated
Character.AI Scaled to 22 Employees With 21 Engineers
“So, so far, 21 of the 22 are engineers.”
Insight
Transformers Beat Recurrent Models by Processing Whole Sequences at Once
“The magic of transformer kind of like convolutions is that you get to process the whole sequence at once. I mean, it still talks, you know, it's still a function of like the, you know, the predictions for the later words are dependent on what the earlier words…”
Disclosure
Character.AI Prioritizes Model Memory So Chatbots Can Remember Users
“Some of the things we want to work on the most are like memory cause you know, our users definitely want their virtual friends to remember them.”
