Shazeer: Naively Training Models on Raw Chat Logs Risks Leaking User Privacy
Noam Shazeer · Noam Shazeer: How We Spent $2M to Train a Single AI Model and Grew Character.ai to 20M Users | E1055 · 20VC with Harry Stebbings · Aug 31, 2023 · at 20:36
Noam Shazeer, co-founder and CEO of Character.ai, explains why AI developers cannot directly train models on unredacted user conversation transcripts.
0:00 / 0:28exact quote · 28.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“It's very important because, like, you know, if you just were to do the naive thing of take, like, every conversation you had with everyone and just train on it, then You could just be, like, spitting out, like, someone's private, you know, private life to, you know, to somebody else. Like, people talk to these, you know, talk to these bots and, like, you know, pour out their heart, you know, like you wouldn't, you want, wouldn't want to just make that, like, freely shareable with everybody.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Noam Shazeer
Shazeer: Character.ai Rejects Product Managers Who Push for Vertical Specialization
“And like, you know, we talked to like some you know, potential product managers early on and they all say the same thing. Oh yeah. Pick your verticals, narrow it down to me, to make it usable. And like, no, we're not going to hire these people. That's like the…”
Shazeer: Users Prefer Video Game Characters Over AI Psychologists for Therapy
“We put up as an example, like a psychologist character, like maybe, maybe you want to, you know, talk to something and feel better, you know, and you know, that gets a little bit of use, but then what we hear a lot more from users is like, I'm talking to a vid…”
Shazeer: Total Computation Is the Single Most Critical Factor in AI Training
“We can get a lot of, we can get a lot of data, but like, really, actually the number one thing that's important is how much computation you do to train it.”
Shazeer: AI Hardware Advances Will Enable Garage Developers to Match Big Tech
“The hardware is progressing so fast that what you could do at a big company, you know, one year, a few years later, you're going to be able to do You know, at the university lab or in your garage.”
Shazeer: Character.ai Views AI Model Hallucinations as a Feature, Not a Bug
“We consider them a feature or at least, okay, basically, our goal, you know, our strategy is, like, launch something general, let people do what they want with it, and if these models are hallucinating, which we, which they certainly are, and we advertise that…”