GPT-3, every mention

mentionsepisodesper episode tap a year for its mentions

Verbatim, from the transcripts: the passages where GPT-3 comes up

loading…

▶ 3:14 Pedro Franceschi Well, I'll tell you my encounter with LLMs, which was so, so I remember in the pandemic, um, there was someone, someone gave me an API access to GPT-III and, uh, and I was playing with it

▶ 16:37 Diana Hu In parallel, GPT-III, was coming out and starting to show signs, until the chat GPT moment around the end of the year, and the industry took off with that, and this was one of the bench work that was really performing really badly, and it…
▶ 18:08 François Chollet And, I mean, it was true of, uh, the original, like, GPT-III actually scoring zero, but that's even true of the latest basal alarms today, you know, as of, as of March.

▶ 2:33 Nick Joseph Yeah, so I was on one of the safety teams, and kind of worked on, ah, I ended working on code models, actually, and kind of, when I got there, I could, the first thing I saw was, oh, they'd fine-tune GPT-III to write some code, but I, and…
▶ 10:16 Nick Joseph Um, like the public estimates for GP three, I remember, were that it cost five million dollars to train, which you're like, on the one hand, five million is kind of a lot, but it's like a lot for an individual person.

▶ 2:51 Jared Kaplan This is actually from the playground of the original GPT-III model, and you can see that as a speaker at a journal club, you're probably elephant me to say certain things.
▶ 6:25 Jared Kaplan So, uh, a researcher about four years ago decided to study scaling laws for AlphaGo, basically putting together two very, very high profile AI successes, GPT-III and scaling for pre-training and AlphaGo.

▶ 24:34 Satya Nadella If there is one thing that I feel, to me, one of the demos I saw, which completely really blew me away, was I think in the beginning of 23, um, when I was in India, and I saw a local developer, Daisy Chain, essentially at that time, either…

▶ 38:09 Sam Altman Almost just about five years ago, like pretty much this week, we put GPT-III into an API and people started playing with it.

▶ 23:27 Michael Truell I mean, GPT-III had happened the year before. 3 times in the scene
▶ 25:12 Michael Truell So we knew, we knew a little bit about this, the internal inside baseball of building GitHub Copilot, the first version, the, the whole building GitHub Copilot story from what I understand, and you know, don't have firsthand knowledge, so…

▶ 8:19 Aravind Srinivas This was even pre-GPT-III.
▶ 12:08 Aravind Srinivas And, um, it was very slow, and it was built with the 175 BGP three model, so incredibly slow and inefficient.

▶ 4:34 Bob McGrew And then, uh, after GPT-ONE seemed successful, we brought in the ideas from Dota and from the robot hand of training at, you know, larger and larger amounts of scale and training on a really diverse set of data, uh, and looking for…

▶ 24:59 Sam Altman Then a few years later, GPT-III happened. 4 times in the scene

▶ 17:59 Diana Hu Like, the older GPT, I forget which one was, like, the older, 2.5 or three, was sufficient in actually creating good enough results, because the vocabulary for a specific domain for hardware or software is a lot smaller than the human…

▶ 7:02 unnamed speaker They'd been experimenting with AI since 2016, but getting access to GPT-III and GPT-IV from OpenAI was the moment that allowed them to create their mega product, CoCouncil.

▶ 2:38 Raza Habib And so where we are today is that, you know, I think starting from GPT one and two, but then GPT three was really the one that I think everyone said, okay, something is very, very different here.
▶ 6:03 Raza Habib GPT-III model and people preferred that despite the fact it was a hundred times smaller.
▶ 8:33 Raza Habib Everyone can use GPT-III.
▶ 14:29 Raza Habib And so the main barrier to replicating something like GPT-III is can you get enough compute and can you get smart people and can you get the data?

Turn any episode into a week of clips.