Everything Stanislas Polu said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Polu: The bulk of useful enterprise agent work can use APIs
“The bulk of the useful stuff that you can do within the company can be done through API. The data can be retrieved by API, the actions can be taken through API.”
Polu: DeepMind IMO breakthrough relied on scaling RL and autoformalization
“I think the DeepMind team just did a good job of scaling. I think there's nothing too magical in their approach, even if it hasn't been published as a Dan Silver talk from seven days ago, where it goes a little bit into more details. It feels like there's noth…”
Polu: Anthropic split was driven by disagreement over OpenAI's API commercialization
“What I understood of it is that there was a disagreement of the commercialization of that technology. I think the focal point of the disagreement was the fact that we started working on the API and wanted to make those models available through an API. Is that …”
Polu: Post-hyper-growth tech companies may increasingly eliminate traditional SaaS
“So it's interesting that we might see kind of a bad time for SaaS in post-hyper-growth tech companies. So it's still a big market, but it's not that big, because if you're not a tech company, You don't have the capabilities to reduce desk cost. If you're a hig…”
Polu: The next generation will see billion-dollar companies with 20 engineers
“All generations of company might be the first billion dollar companies with engineering teams of 20 people. That would be so exciting as well. That would be so great. You know, you don't have the management hurdle. You're just 20 focused people with a lot of a…”
Stanislas Polu: If transformative AI arrives in 5 to 10 years, OpenAI will likely build it
“But if it takes five years or 10 years, then that's probably where it's gonna happen.”
Polu: A single all-knowing enterprise AI assistant is currently far-fetched
“Context size and model quality makes it such that an assistant that knows it all within a company is, is, is, is still kind of a bit farfetched because it'll get confused if it sees too much information.”
Polu: Building a strong AI research team is 10x easier in Paris than SF
“If you want to build a strong AI research team today, it'll be 10 times easier to do it in Paris than it is to do it in SF with OpenAI as a lab, as a competitive lab to, in the same hiring markets.”
Polu: OpenAI Managed Research Priorities Directly Through Compute Allocation
“In that space, there's a managing tool that is great, which is computer location. Basically, by managing the computer location, you can message the team of where you think the priority should go. And so it was really a question of you were free as a researcher…”
Polu: OpenAI Believed in Transformer Scaling Pre-Kaplan Paper
“Before that, there really was a strong belief in, in scale. I think it was just the belief that the transformer was a generic enough architecture that you could learn anything, and that this was just a question of scaling.”
Polu: GPT-4 was ready internally at OpenAI months before September 2022
“I had seen GPT-IV internally at the time. It was September, 20, 22. So it was pre-chat GPT, but GPT-IV was ready since, I mean, I'd been ready for a few months internally.”
Polu: Fully autonomous AI models 'get lost' and are not ready
“The AutoGPD approach, obviously, is extremely exciting, but we know that the agentic capability of models are not quite there yet. It just gets lost.”
Polu: Hierarchies of simple agents will unlock Auto-GPT level value
“Once you have those working really well, you can create meta agents that use the agents as actions, and all of a sudden you can kind of have a hierarchy of responsibility that will probably get you almost to the point of the auto GPT value.”
Polu: GPT-4 Turbo performs better than GPT-4o on function calling
“I personally don't have proof, but I know many people, and I'm probably part of them, to think that GPT-IV Turbo is still better than GPT-IV on function calling.”
Polu: Claude Sonnet executes an unpublicized chain-of-thought step during function calling
“They kind of innovated in an interesting way, which was never quite publicized, but it's that they have that kind of chain of thoughts step whenever you use a Clouds model or Sonnet model with function calling. That chain of service step doesn't exist when you…”
Polu: Airbyte's Notion connector output is not useful for AI models
“And the reality is that if you look at Notion, Airby does the job of taking Notion and putting it in a structured way, but that's a way that is not really usable to actually make it available to models in a useful way. Because you get all the blocks, details, …”
Polu: Vertical AI agents have easier GTM but limited enterprise upside
“Vertical solutions have a good market that is much easier because they're like, oh, I'm going to solve the lawyer stuff. But the potential within the company after that is limited. So there's really a nice tension there. We, we're true believers of the horizon…”
Polu: Compute allocation naturally aligns AI researchers with company goals
“There is a way to Orion's an organization, a research organizations, the way you allocate computes. Which means that as a researcher, it's often the case that you are free to work on whatever the things you want to work on. Right. But if the things you're work…”
Polu: Onboarding enterprise users to AI requires constrained tools, not general assistants
“For a large amount of the people within companies, The best way to onboard them the technology is to not give them a very general assistant that can do anything and everything, but instead focus the assistant to some use case that feels more like a tool.”
Polu: Enterprise AI adoption will scale via product, not custom services
“I think all of that can probably scale through product and not necessarily handholding and kind of custom work with our clients eventually.”
Noisy retrieval causes even state-of-the-art LLMs to skip relevant context
“If what you're searching is too large, the answers, the chunk that you'll be finding will be a bit noisy, and the models, even the best ones today, they have a tendency to get a little bit disturbed by that noise, meaning that they might not hallucinate too mu…”
Polu: Enterprise AI needs frontier models rather than complex query routing
“Most of the tasks are pretty general, right? Most of the tasks are pretty like a human would do. And so you just want the best models. And as it happens today, the best models are before enclosed. So that's what you want.”
Polu: Dust is near repeatable sales but has not reached PMF yet
“I think we are in the, we are on the precipice of repeated sales processes. So that could call PMF. I wouldn't quite qualify it as PMF yet.”
Mistral and Poolside funding is a trickle compared to OpenAI, says Polu
“It's already awesome that Mistral was able to raise that much, that Toolside is able to raise that much, but it's a trickle compared to what Open Air is raising, compared to what Anthropik is raising.”