Shazeer: Impressive Language Models Existed Unreleased in Labs by 2020
Noam Shazeer · Noam Shazeer: How We Spent $2M to Train a Single AI Model and Grew Character.ai to 20M Users | E1055 · 20VC with Harry Stebbings · Aug 31, 2023 · at 16:25
Noam Shazeer, former Google Brain researcher and co-founder of Character.ai, discusses the timeline of neural language model development preceding the public AI boom.
0:00 / 0:10exact quote · 10.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I'd say around 2020 was when sort of, like, really impressive stuff was sort of in the lab, but not launched.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Noam Shazeer
Shazeer: Character.ai Rejects Product Managers Who Push for Vertical Specialization
“And like, you know, we talked to like some you know, potential product managers early on and they all say the same thing. Oh yeah. Pick your verticals, narrow it down to me, to make it usable. And like, no, we're not going to hire these people. That's like the…”
Shazeer: Users Prefer Video Game Characters Over AI Psychologists for Therapy
“We put up as an example, like a psychologist character, like maybe, maybe you want to, you know, talk to something and feel better, you know, and you know, that gets a little bit of use, but then what we hear a lot more from users is like, I'm talking to a vid…”
Shazeer: Total Computation Is the Single Most Critical Factor in AI Training
“We can get a lot of, we can get a lot of data, but like, really, actually the number one thing that's important is how much computation you do to train it.”
Shazeer: AI Hardware Advances Will Enable Garage Developers to Match Big Tech
“The hardware is progressing so fast that what you could do at a big company, you know, one year, a few years later, you're going to be able to do You know, at the university lab or in your garage.”
Shazeer: Character.ai Views AI Model Hallucinations as a Feature, Not a Bug
“We consider them a feature or at least, okay, basically, our goal, you know, our strategy is, like, launch something general, let people do what they want with it, and if these models are hallucinating, which we, which they certainly are, and we advertise that…”