OpenAI

“Other things that, you know, I'd say we try to avoid is things that are roadkill along the way of a company like OpenAI or Google or Amazon or Meta. So, small refinements into what the model doesn't do well today. That's just gonna be real cool along the way.”

“The second that DeepSea comes out, it suddenly accelerated the internal strategic decisions from OpenAI of when are we gonna release models.”

“At the end of the day, we are accountable for the safety of the systems we produce. Not just OpenAI, like every, every engineer and that's a principle that should not change.”

“What at OpenAI, what we did with the O-one model, which is to do some reinforcement learning those chains of thought to really reach new levels of intelligence.”

“I hadn't been paying attention to large language models. I just, it didn't follow the progress after the Transformers paper. And I saw that and my reaction was, I had no idea computers could do that.”

“I often, with the reasoning models, which are slower, will use a faster model first, GBT-IV-O, to refine my prompts.”

“I haven't done the math on it, but it's probably cheaper to use that than to host, self-host any of the open source models.”

“Is I do think that, It will be the delivery mechanism for AGI when it's produced, and not just because of the many researchers at OpenAI, but because of the amazing, like, utility it's become from individuals.”

“Sam Altman probably brought forward the future by, you know, five years, 10 years, at least 10 years.”

“I mean, I think Microsoft has you know, just really, really deep access to OpenAI, and I think OpenAI is probably, you said public companies, right? ... So, you know, I think There's a non-zero, pretty large percentage of, like, the market cap of Microsoft tha…”

“We'll see fewer, what are known as foundation model companies, the people building the open AIs or Anthropics or some of the Google core technologies.”

“As a search replacement, so I tend to use Claude you know, or Gemini, or OpenAI, where I would have used search. So, in the ski industry, I've been learning a lot of things about lifts, and lift, you know, cable strengths, and other things like that, and water…”

“You have that, the OpenAI and people have been doing memory, where it remembers what else you've asked, but that seems more like a switching cost than a network effect, and also it might be easier for you to just ask it what it knows about you and then tell Cl…”

“There's no apparent equivalent in LLMs right now. There's no reason why the LLMs get better because more people use them.”

“Now, OpenAI is doing weekly active users, and Sam Waltman was a social media startup founder. He knows this. It's a bullshit number.”

“They're not retraining the models all the time with the data. So you don't have that kind of runaway effect as, like, continuous flow and more queries produces, you know, better results.”

“Microsoft has this kind of weird situation in that his own models aren't actually very good, but it's got this very kind of weird relationship with OpenAI.”

“OpenAI Sam Altman is, I was going to say polarizing figure, but actually opinions about him tend to be fairly unanimous and tend to be fairly negative, like everybody who's ever worked with him quit.”

“NVIDIA invests in OpenAI, and NVIDIA is a supplier of chips to OpenAI. So it's all circular, what's going on here.”

“On the one hand, just predicting what comes next sounds like a pedestrian task, but if you really can predict the next word out of Einstein's mouth, you are at least as smart as Einstein.”

“If you look at math and physics, we now are solving open math problems. We're solving Open physics problems and actually have resolved this particular physics problem recently in quantum physics in the opposite way that the community expected.”

“Fundamentally the technology that we use to train during unsupervised stage and during the reinforcing stage, they're exactly the same. You are just predicting, but you've changed the structure of the data.”

“Initially, the set of people that I narrowed down to were actually Ilya, Dario, Amidai, Chris Ola, and myself. That was going to be the team.”

“I think that this kind of thing happening this year is not out of the question.”

“We should be in a world, if we do our job right, where everyone gets access, has a doctor in their pocket that is better than any team of doctors today, the world's best doctors, they're there for you, they care about you, they're actually reading your charts,…”

“If you truly believe in the possibility of creating machines that have the intelligence level of humans, It means the stakes always feel very high. The question of who's making the decision, the question of what are the values that go into those decisions, the…”

“It's hard to know what percent of the code is not written by AI. It's a vanishing fraction. The actual Writing of code. Currently, the AI is much better than humans at writing code, given the right context, given the right structure. Now there's parts of the a…”

“People are getting offers and we actually did not lose a single person through that weekend. No one accepted a competing offer.”

“And we're going to be hitting a phase soon where the AI will also come up with its own research ideas and test those out, run experiments.”

“We've actually gone through an evolution of how we train the models to user preferences, and that we've seen that at one point, like last year, that the models really did start to lean into telling you what you wanted to hear, saying, oh, that's such a great a…”

“There's certainly a lot of attempts to distill models. And that comes from companies in the U S it comes from all over the world. But I think that it misses the core point, which is that the way this technology is developing is it is on an exponential. And any…”

“Sam talked to Satya, who we'd been talking about. Hey, can you be a funder? Could you know, help, help support this new endeavor? I was like, Hey, actually, can we expand from the small life raft to like, yes, can we take everyone? And we're like, all right, w…”

“We came up with what I would Really say is almost the technical plan that we have pursued for the past 10 years. Number one, solve reinforcement learning. Number two, solve unsupervised learning. And number three was gradually learn more complicated, in quotes…”

“And honestly, just one of the hardest moments for me at OpenAI was when Ilya left. And it was maybe the only moment in OpenAI's history where I felt like I didn't want to do it anymore.”

“We've had many issues in the past where the cables were just too taut, just literally like too, too tight of cables. And then you get signal integrity issues and the computer doesn't work.”

“You talk to a doctor, you talk to a lawyer, those are privileged conversations, right? You feel comfortable sharing them. There's certain guardrails on when the healthcare provider would have to, you know, provide that information to law enforcement or alert s…”

“If you just wanted enough compute for, you know, you wanted one GPU for every person in the world, you're talking like eight billion GPUs. We are not on a trajectory to build anywhere near that level of compute, right? It's like, you know, hundreds of thousand…”

“And you think about things like data centers, that those are something where there's clearly been a lot of concern about questions like, do they drive up electricity prices? And we have a commitment to ensure that they do not.”

“So many people were trying to sign the petition at once, it actually crashed Google Docs. And so you had to have certain people who were designated as the person you go to actually put your name on the document so you don't have too many editors at once.”

“There were a few of my close collaborators who quit that day as well. That's Jakob, Shimon, Alexander, and the five of us. So Those people plus Sam. We all got together and we started to chart out what a new company could look like.”

“So there's two reasons. One is to think about distillation, but the second, in some ways more important, is that we had this insight when we first developed the reasoning paradigm that it gives us a interpretability mechanism we had not been anticipating, beca…”

“Because it's 2017, and it's really the first time that we saw semantics arise from training on A language modeling objective. So you train on learn the next character, predict the next character, and then suddenly you get a neural net that understands sentimen…”

“Elon, Sam, Ilya, and I all agreed. That the only path forward for OpenAI, the only path to achieve the mission, was to create a for-profit entity associated with OpenAI of some form. And so we were committed to that direction, and that is something that we kne…”

“I was told that the board has decided That Sam would be removed, and effectively the message that I got was the same messaging that was in the public post, and I asked if I could have any more information. I was told no, not right now, and I pressed on that ma…”

“It was medical spam, like advertising different drugs to people, right? It's like not something we ever would have thought of as a problem.”

“Like a good example is data centers and water usage. Like that, that's something that people talk about a lot, but actually our data centers use incredibly little water, right? That's actually misinformation that they use a lot.”

“And by the way, the neural net we used, tiny, tiny little insect brain, similar number of synapses as to truly an insect brain.”

“We're all going to be heading to a world where we're managers of agents and soon maybe The CEO of an autonomous AI corporation, right?”

“We didn't go to any other VCs or anything like that.”

“On 86 out of a hundred of those questions, three out of three attorneys said yes, and that was the Oh my God moment for me and for my co-founder, or so we need to do something in this industry.”

“I think that there are workflows and data moats that if you get, and also just understanding, like, there's three or four legal startups in the AI space. They're just spending so much more time making sure they ingest all the case law and really understand, yo…”