The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 8 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Tworek: OpenAI team was initially underwhelmed by pre-trained GPT-4
“When we trained GPT-IV, we were pretty underwhelmed internally, and then there was a lot of moments, oh, we trained this small, we spent a lot of money on it, and it's kind of like, you know, pretty dumb, at least, like, you know, we have GPT-IV, GPT-III alrea…”
Writer relied on GPT-3 for only a brief three-month period
“So we, we've always had our own technology down to the LLM. There was a very brief period, literally like three months, where we, for some of our apps, used GPT-III because it was just so much better.”
Fine-tuning a GPT-3 class model with LoRA provides a 10,000x efficiency boost
“I think for something like a GPD three level model, it's a 10,000 X speed up in efficiency while losing nearly not much at all in accuracy.”
Zhou: Fine-tuning transformed GPT-3 into ChatGPT
“Fine tuning is the technology that got from a research project in 2020 called GPT-III and turned that into ChatGPT, a billion dollar app, right?”
Falcon: PyTorch Lightning's original tooling was insufficient for post-GPT-3 architectures
“The tool that we had was good for 2016 through 20 19 deep learning, but as of GPT three deep learning, it wasn't good enough, and so we had to upgrade to this kind of new paradigm that we introduced.”
Mann: GPT-3 was vastly better than decades of prior NLP systems
“Two years ago, I remember being shocked at GPT-III when it was released, because it was just so, I'm an old NLP guy, and it was so vastly better than anything that, that we could build for decades.”