The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 132 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 1 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Hu: OpenAI o1 Combines Next-Token LLMs with RL Reward Functions
“GPT is all generative based on predicting the next token and patterns and then getting those results to check that they're correct. So I think a lot of it is you had to have a lot of data that was factually correct and Fed into probably the model and the train…”
Altman: Conviction is essential until empirical data emerges
“Conviction is great until the moment you have data one way or the other. And there are a lot of people who hold on it past the moment of data.”
Altman: OpenAI's GPT Series Originated From Radford's Unsupervised Sentiment Neuron Discovery
“And at the time, unsupervised learning was just not really working. So he noticed this one really interesting property, which is there was a neuron that was flipping positive or negative with sentiment. And yeah, that led to the GPT series.”
Altman: OpenAI initially focused on robotics and gaming agents over LLMs
“I had no idea that language models were going to be the thing. You know, we started working on robots and agents playing video games and all these other things.”
Altman: OpenAI's original goals included never exceeding 120 people
“I believe there were three goals for the For the effort at the time. It was like, figure out how to do unsupervised learning, solve RL, and never get more than a 120 people. Missed on the third one.”
Friar: OpenAI's capital spending prioritizes compute first, then top talent
“It's compute first and it's not cheap. It's great talent second. And then of course it's all the normal operating expenses of a more traditional company.”
Harj Taggar: Competing with OpenAI on cost alone fails to retain customers
“What I think a bunch of the companies in the space are seeing is that, like, that's not enough to keep the customers, especially because, like, OpenAI, like, the cost of all of the models is just going down.”
Habib: Fine-tuning is what separated ChatGPT from earlier GPT models
“If you look at what the difference is between ChatGPT or the most recent OpenAI Text DaVinci Three model, and what's been in the platform for two years and has not gotten as much attention, the difference is fine tuning. Like it's the same base model, more or …”
Altman: OpenAI is one of the most interesting research labs in decades
“And I do think we've done one of the more interesting research labs of the last couple of decades. The world isn't set up to support this anymore, and I, and it's a very interesting question about why.”
Brockman: ML researchers copy files over Git because experiments must run side-by-side
“Because the thing is, if you have a new idea for, okay, well, I've kind of got this thing working, and now I'm going to try something slightly different. As you're doing the new thing, well, machine learning is, to some extent, very binary. At the start, it ju…”
Sidor: OpenAI's Dota bot required daily overnight updates to beat pros
“Because actually it's a little known fact, but from day to day, like each version of the experiment was not good enough to beat the next player, next, next day's professional.”
Brockman: OpenAI character-prediction model learned state-of-the-art sentiment classification
“At OpenAI, we see this sometimes, for example, we had a paper on this unsupervised learning where you train a language model You train a model to predict the next character in Amazon reviews, and just by learning to predict the next character in Amazon reviews…”
Brockman: OpenAI chose Dota 2 after League lacked Linux support and API
“Actually the way that we selected the game was we went on, on Twitch and just looked down the list of most popular games in the world and starting, you know, number one is League of Legends. The thing about League of Legends is it doesn't run on Linux and it d…”
Altman: AI PhD not required to work at OpenAI
“Everyone thinks they have to be an AI PhD. Not true. Neither of these guys are.”
Sam Altman: Helping people unconditionally consistently yields massive career upside
“Whenever I've helped people for no immediate benefit and with no intention of ever getting a benefit at all time and again in my career, it has really Later, benefited me a lot.”
Altman: Met co-founder Brockman while recruiting him to Stripe
“I met Greg Brockman my co-founder, OpenAI, because I was a very early investor in Stripe when I was, like, I don't know, 22 or something, and they were, like, this was before they had, you know, like, real investors that could really help them, and they said, …”
Chou: Ploy uses 3,500 curated web design prompts for inspiration
“We've used a collection of models, open AIs, ChatGPT images, but then we've created 3500 prompts for web designs that then Ploy takes inspiration from.”
Mellata: OpenAI model upgrades cut Variance's pilot costs tenfold
“As we were running this pilot for this first customer, sort of OpenAI was coming out with new models in the middle of the pilot, which was changing, one, our cost structure by a 10 x factor, but also was changing our performance quite a lot.”
Nick Joseph followed OpenAI safety leads to join Anthropic at founding
“Basically everyone I worked with, like all the safety leads left, which yeah, invited me to go to Anthropic, and that was sort of the reason I joined OpenAI, was because I cared about AI safety and wanted to work with them. So then I went with them to join Ant…”
Altman: OpenAI started with no product ideas, revenue, or revenue expectations
“We were trying to, like, play video games, and we had this little robotic hand that could sort of barely do a Rubik's Cube, and we had no ideas for products, no revenue, no really idea that we were ever going to have revenue, and we were, like, sitting around …”
Altman: Founding team came very close to not starting OpenAI
“Like, there, we got very close to not starting OpenAI.”
Truell: OpenAI's original Codex model cost roughly $100,000 to train
“Is we did the back of the envelope math for what Codex, the first coding model, costed to train. From my memory, it only cost about 90 K or a hundred K by our calculations.”
McGrew: Aditya Ramesh spent up to two years building DALL-E 1
“I think Aditya Ramesh, who built that model, spent 18 months, maybe two years, just getting to the first version that, that clearly worked.”
Blomfield: Startups must treat cloud and AI credits as cash costs
“So people at AWS, Microsoft, OpenAI, giving out tons and tons of credits to startups. You should treat those as a cash cost. Don't assume you'll have unlimited credits forever. It'll totally mess up your margins.”
Heller: Casetext received GPT-4 access six months before release
“When we got the chance to work at GPT-IV, maybe about six or so months before it was publicly released,”
Habib: OpenAI's 1.3B InstructGPT beat the 100x larger GPT-3 via RLHF
“In the InstructGPT paper that OpenAI released, they compared, you know, a one or two billion parameter model with instruction tuning and RHF to the full GPT-III model and people preferred that despite the fact it was a hundred times smaller.”
Brundage: FHI relies on DeepMind and OpenAI ties, lacks major compute
“At the Future of Humanity Institute, we have a lot of relationships with organizations like DeepMind and OpenAI and others but, you know, we don't have, like, a ton of, you know, GPUs or TPUs here like, running the latest experiments outside of, you know, some…”
Brockman: OpenAI Dota bot lost to Pajkatt due to an unseen wand build
“Well, I, I'd say very, very specifically that kind of the root cause here was that he had gone for an item, an early wand build. And we had just never done item early wand build. And so it's just like our bot had just never seen this particular item build befo…”
Sidor: OpenAI ran 50-computer LAN party where humans found bot exploits
“There was a point after the event where we set up this big LAN party where we had, like, 50 computers running the bot. We kind of unleashed this swarm of humans to kind of add our bot. And they find, found all the exploits”
Brockman: OpenAI Dota bot went undefeated 5-0 against Sumail
“With Sumail, we went undefeated. I, and I think it was five zero that day.”
Brockman: OpenAI's hybrid bot defeated pro player Arteezy 3-0
“We first played our TZ who showed up on, on our switch bot, you know, kind of the Franken bot. And you know, that beat him three times.”
OpenAI downloaded 2TB of Dota 2 replays daily before filtering
“Originally, we were downloading all of them every day, and realized that was about two terabytes worth of data a day. That adds up quite quickly, so we ended up filtering down to the sort of most expert players.”
Brockman: OpenAI stitched two bots together hours before facing Dota pros
“Monday bot, it is pretty good at the first wave. This new bot is a super bot thereafter. ... So we wrote, so, so we already, we had some code for doing something similar. So we kind of revived that. And then in the three hours, Jay spent his time doing a very …”
Brockman: OpenAI put all compute into one final run before tournament
“Two weeks before the International was kind of our cutoff for at this point, there's not much more we can do, that we're gonna do our biggest experiment ever, put all of our compute into one basket, and see, see where it goes.”
Zaremba: OpenAI focuses robotics research on object manipulation
“So in terms of robotics, we are working on manipulation. We think that manipulation is the complete, it's the one of the parts of robotics, which is the most unresolved.”
Zaremba outlines OpenAI's mission to develop safe, beneficial AGI
“So OpenAI focuses on building AI for the good of humanity. We are a group of researchers and engineers collaborating together who essentially try to figure out what are the missing pieces of artificial, of general artificial intelligence and how to build it in…”