The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 25 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Srinivas: Google possessed only fourth or fifth best AI models in 2023-2024
“And 2024, 20 23 especially, and large part of 2024 too, Google had like, maybe a fourth or fifth best models at any moment. So as a startup outside Google, you had access to AI that was better than what Google internally had.”
Elon Musk emailed OpenAI saying it had zero percent success chance
“I can say this one because it got publicized in early, not early, a few years into OpenAI, where Elon sent us this really mean email. We'd been working through that for a while and said we had a zero percent chance of success. Like, not 0.10, that we were tota…”
Srinivas: OpenAI's built-in search features in ChatGPT did not kill Perplexity
“OpenAI has perplexity within ChatGPT that did not kill any of these companies.”
Tan: OpenAI Uses Fake Model to Hide Raw o1 Chain of Thought
“If you use O-one in ChatGPT, it looks like it will tell you what's really going on, but apparently they have a fake model that just spits out things to give you the impression that it's breaking it up into steps. And they've actually You know, hidden it, becau…”
French-Owen: Codex Uses Periodic Compaction to Support Long-Running Tasks
“Where it will run compaction, like, periodically after each turn, and so Codex can continue to run for a very long time, and if you look at the percentage in the CLI, you'll see it, like, move up and down as compaction runs.”
Chollet: Fine-Tuned OpenAI o3 Reached Human-Level Performance on ARC
“So in particular, in December last year, OpenAI previewed its, ah, all three model, and they used a version of it that was, ah, fine-tuned specifically on Arc, and that showed human-level performance on that benchmark versus time.”
Altman: I wrote OpenAI's early checks before Elon Musk funded scale
“It was definitely helpful that I could just, like, write the early checks for OpenAI, and I think it would have been hard to get somebody else to do that at the very beginning. And then Elon did it a lot at a much higher scale, which I'm very grateful for, and…”
Tan: OpenAI enabled internal model distillation as a developer lock-in strategy
“OpenAI itself has now enabled distillation internal to its own API. So you can use O-one, you can use even GPT-IV or IV-O to distill it down into a much cheaper model that's internal to them, like GPT-IV, IV-O-Mini. And that's sort of their, you know, lock-in …”
Sidor: OpenAI's RL bot beat its hand-coded bot after one to two weeks
“So I leave, there is nothing, I come back, there is this reinforcement learning bot, and actually, it's beating our scripted bot after, like a week worth of engineering effort. Possibly it was two weeks, but it was something very miniature compared to the deve…”
OpenAI Dota bot learned baiting behavior without explicit reward incentives
“It was like one of the major examples, one of the major examples of the things that we kind of didn't have explicit incentive for, and yet the bot actually learned them.”
Zaremba: OpenAI has secured $1 billion in total investment
“In total we gather an investment of one billion dollar in the group.”
Musk in 2016: OpenAI is structured as a 501(c)(3) nonprofit
“OpenAI is structured as see a five one C three nonprofit but, you know, many nonprofits do not have a sense of urgency.”
Altman: OpenAI's o3 model cost dropped by 5x in one week
“And also, like last week, O-three cost five times as much as it did this week, and that's gonna keep going.”
McGrew: OpenAI authored early robotics paper as 'OpenAI' to avoid credit disputes
“And one of the early robotics papers, we actually said site as open AI because we didn't want to get into a fight. You know, the first author is the one who, you know, gets cited and their name shows up every single time. So we said, you know, we're not going …”
Friedman: Full OpenAI o1 Model Is a Huge Step Function Above o1-Preview
“Like the full O-one model, which is coming out any day now is a huge step function above even O-one preview, which is what enabled all these incredible results at the hackathon.”
Altman: OpenAI's GPT Series Originated From Radford's Unsupervised Sentiment Neuron Discovery
“And at the time, unsupervised learning was just not really working. So he noticed this one really interesting property, which is there was a neuron that was flipping positive or negative with sentiment. And yeah, that led to the GPT series.”
Habib: Fine-tuning is what separated ChatGPT from earlier GPT models
“If you look at what the difference is between ChatGPT or the most recent OpenAI Text DaVinci Three model, and what's been in the platform for two years and has not gotten as much attention, the difference is fine tuning. Like it's the same base model, more or …”
Brockman: OpenAI character-prediction model learned state-of-the-art sentiment classification
“At OpenAI, we see this sometimes, for example, we had a paper on this unsupervised learning where you train a language model You train a model to predict the next character in Amazon reviews, and just by learning to predict the next character in Amazon reviews…”
Altman: AI PhD not required to work at OpenAI
“Everyone thinks they have to be an AI PhD. Not true. Neither of these guys are.”
Heller: Casetext received GPT-4 access six months before release
“When we got the chance to work at GPT-IV, maybe about six or so months before it was publicly released,”
Habib: OpenAI's 1.3B InstructGPT beat the 100x larger GPT-3 via RLHF
“In the InstructGPT paper that OpenAI released, they compared, you know, a one or two billion parameter model with instruction tuning and RHF to the full GPT-III model and people preferred that despite the fact it was a hundred times smaller.”
Brockman: OpenAI Dota bot lost to Pajkatt due to an unseen wand build
“Well, I, I'd say very, very specifically that kind of the root cause here was that he had gone for an item, an early wand build. And we had just never done item early wand build. And so it's just like our bot had just never seen this particular item build befo…”
Sidor: OpenAI ran 50-computer LAN party where humans found bot exploits
“There was a point after the event where we set up this big LAN party where we had, like, 50 computers running the bot. We kind of unleashed this swarm of humans to kind of add our bot. And they find, found all the exploits”