The Ledger, every show

“So we, one of the things we did is we published DSM-I-K, which was a held out eval. So we basically produced a new evaluation of the math capabilities of models. That there's no way it would ever exist in the training data set to really see how much of the, ho…”

“So, you know, we got multi-modality capability. That's exciting. It's more of a lateral expansion of the models, and the industry needs smarter models. We need GPT-V, or we need Gemini-II, or whatever that, those models are going to be. and so to me it was, y…”

“Every time we would go out to fundraise, even though our numbers were great and we had great revenue, you know, VCs and investors would always be very skeptical. They'd be like, Oh, I don't know if this is a good business. Does it have longevity? Is this durab…”

“We're gonna keep seeing these new modalities and form factors and developments of the AI paradigm that will each be dramatically bigger than the last.”

“I mean, I think there's still just, like, astronomical opportunity in agentic looping, and figuring out how you develop systems that enable you to spend, like, 1000 X more or one million X more on tokens to drive an outcome in a continuous feedback loop.”

“I think it's actually a pretty bullish case for society adapting the technology because I think it's going to be, you know, consistent, slow progress. For quite some time, and society will have time to fully sort of acclimate to the technology that develops.”

“I do expect that the new admin will come in and help accelerate things to enable the US to compete more aggressively with China and ultimately come out ahead on the technology.”

“I think one of the best examples of this was in the past year, the Biden admin posed to the UAE, hey, which way are you going to go in terms of AI technology? You can either go into the sort of Huawei China stack, or you could go into the Microsoft United Stat…”

“By the way, I think that the drone warfare in Ukraine is becoming more and more Enhanced by generative AI and more advanced autonomy.”

“I mean, right now we're so stuck as a I think tech industry still on the sort of like chat paradigm and, you know having everything be a chat with one of these models. And I think that's a constrictive paradigm to enable agents to actually really start working…”

“Having a product experience, which Where you don't expect it to be perfect, but you expect it to be pretty good. I think that's like, 99% of the challenge, and that's why we haven't seen it yet, despite the fact that the models already can do a lot of this stu…”

“It's a very sort of regimented way to try to enable the systems to do multi-step reasoning. Cause ideally what you want the model to do is to just like how a human does, be able to sort of go through and figure out what are the bits of pieces it needs to know …”

“I think quantum computing is on kind of like the way AI was back in 2018. It's on a few scaling laws where you can definitely sort of squint and see that, you know, in five to 10 years, this is going to be A really, really impactful technology.”

“We're kind of hitting this wall where we've leveraged all the publicly available data. And so one of the hallmarks of this next phase is actually going to be data production.”

“I think that the real way, way in which a lot of the large labs have just dramatic advantages is just, they have very profitable businesses that can provide Near infinite sources of capital for these AI efforts.”

“If, yeah, Facebook, Google, they make their advertising systems a little bit better. They can recoup billions of dollars just by the... Better performance there. Apple can easily recoup the investments if it drives an upgrade cycle. I mean, these are things th…”

“I think an Anthropix launch of artifacts in Claude is like a it's like the first pin drop of this major theme of, you know all the labs are going to be pushing much deeper product integrations to be able to drive higher quality businesses.”

“There's latent potential for almost every enterprise to implement AI at a level that would meaningfully boost their stock price.”

“I think if most of the benefit is on the cost saving side, then that's not really enough to disrupt large incumbent that has already kind of, you know, there's kind of like push their way through all the costs of gate of growing and distribution.”

“They pay consulting firms tens of millions of dollars, hundreds of millions of dollars to do these data migrations. And it's, you know, even after that, no change in results.”

“So over the past few years we've Basically kept our headcount flat. I mean, we've grown it very slightly as the business grown, but the business itself is, you know, five X, well, six X, like, you know, the business has grown dramatically.”

“If you have a very high performing team and a very high performing org, it's in nearly impossible to grow it dramatically without losing all of that high performance and all of the winning culture.”

“I like the definition of this. That's sort of like, you know let's say 80 plus percent of jobs that you, that people can do purely at computers, so digital focused jobs are accomplishable with, you know, AI can accomplish those jobs.”

“Was the outsourcing of American automotive manufacturing. So you know, We used to have all this domestic automotive manufacturing, then it sort of slowly got outsourced to outside the United States. Over a few decade time period, it resulted in displacement of…”

“Knowing how to interface with AI systems is an incredibly critical capability that is, I think is very akin to software engineering over the past few decades.”

“There's plenty of products like that that exists. They all suck because the models are nowhere close to reliable enough where you can actually depend on it to do any of those activities. And even if it has, let's say even a model has like 90% reliability, whic…”

“What I'm excited about, and I think is actually closer than we think, is the next unlock that we're talking about, where it's like actually autopilot, true autopilot for everything, and that Reliability boost in the models, I think could come here faster than …”

“This is one of the reasons why for the current brand of AI technology, I'm actually much more focused on the B to B and the enterprise applications of the technology, much more so than the consumer applications, because I think in the consumer applications, we…”

“So we built the very first data engines to support government data. This would support mostly geospatial and satellite and over, other overhead imagery. This ended up fueling the first AI program of record for the US DoD.”

“Today, you know, fast forward to today our data foundry fuels basically every major large language model in the industry. Work with OpenAI, Meta, Microsoft, many of the other players.”

“It's not Any more the case that these models can learn that much more from, you know, various comments on Reddit or whatnot. They need, ah, they need truly frontier data.”

“And our perspective is that the critical thing is, is what we call hybrid human AI synthetic data. So how can you build hybrid human AI systems such that AI are doing a lot of the heavy lifting, but human experts and people, you know, the basically best, brigh…”

“GPT-IV, I think, as a model, was a little early of a technology for us to have this entire hype wave around, and I think we, you know, the community very quickly discovered all the limitations of GPT-IV... It was probably a few generations too early of a model…”

“One of the things that every, you know, all the model developers understand well, but the enterprises understand super well is that you know, not all data is created equal and high quality data or frontier data is, is, can be, you know, 10,000 times more valua…”

“Like, I think if you look at all the most successful companies in the world they were started at a time long before the sort of, like, core idea was popular. And they work on that. They toil in obscurity for years and years before, you know, the idea or the sp…”

“Today the easiest way is to use open code. We have, like, onboarding on the website, and then soon we'll have a harness of our own, and ultimately, I think we want great models that plug into all of the available harnesses and empower as much, you know sort of…”

“Systems thinking is never going to go out of style. So I think it's definitely a mistake to go all in on word cell. Like I think you need to shape rotate.”

“Try to identify what is the exponential in the world that has both the steepest curve And we'll go the longest. And, you know, many decades ago, this curve was Moore's law. And that probably was, you know, that was at the time, like clearly the right thing to …”

“I think this is going to be One of the under the line battles that really defines the course of the next few decades of geopolitics.”

“One undeniable trend over the past, let's call it five years has been the sort of the collapse of the Chinese startup sector. And this is really driven by policies from the CCP to significantly, you know, they killed certain startup industries. They really lik…”

“An ideal AI agent is one that, that I think is observing and naturally in all the sort of like core flows of information and core flows of context that you are in digitally. So, you know, it's in all your Slack threads. It's only your email threads that like, …”

“I mean, I think ultimately, I think agents are gonna start In an area that they'll feel pretty it'll feel like a toy, just like with any technology. So maybe you know, we'll all start with like a language learning agent, or we'll start with a cooking aid agent…”

“And I think what's going to happen in 25 is we're going to we're going to only be focused on who can create newer, better chips or bigger data centers with more chips, but also who can create newer and better data. And one of the things that I think we're goin…”

“One of the things that really is true in all the models today is that they're not that good at, you know at taking multi-step actions. Whenever it has to take a few hops, whenever it has to take, chain a few things together, it'll invariably make mistakes alon…”

“One thing that we see today with the models is that because all the benchmarks that were used today are what's called saturated, i.e., you know, in other words, like all the models do really well at the benchmarks, it's really hard to discern actually which on…”

“Yeah, I think it's I think this will be one of the great human projects of our time if that makes sense.”

“For the past two-ish years or the past maybe three, four three years, let's say three years it's almost been more about execution than anything. It's a lot of just engineering, like how do you actually have large scale training work well. How do you make sure …”

“We're gonna have to get pretty scientific around, You know, exactly what is the model not capable of today? And therefore, what are the exact kinds of data that need to be added to improve the model's performance?”