The Ledger, every show

“And then there's a new story that's emerged over the last like five months, which is, oh, we're going to scale up this test time compute and that's going to get us to AGI. And I think what V two shows is that that's not quite either. We still need some structu…”

“Language models. Generally working like a memorization style regime where they're right. Learning lots of data. They're able to apply it to very similar types of patterns that they've seen before, but not novel patterns. That's what RKGI shows.”

“You know, my belief is that AGI progress has really stalled out over the last four or five years.”

“Arc AGI to best of my knowledge is the only true. AGI eval that actually exists in the world and measures a actually good definition, correct definition of what AGI is, which we can talk about.”

“Effectively what large language models do today is they are high dimensional memorization systems, right? They are trained on lots of training data. They're able to find and generalize patterns off of the training data that they're trained on and then apply th…”

“Scaling language models purely will not get there.”

“I think it's incredibly dangerous to try and make predictions about future capabilities, about where the technology will go and make rules, legislations, laws. Like prohibiting or enforcing or requiring certain research directions through a theoretical lens. I…”

“It's quite likely actually that the solution it can be like written in like 10,000 lines of code or less. And it's not gonna require these like, you know, gigantic You know, two hundred billion large parameter models in order to solve it.”

“Surprise that it basically hadn't, and not only hadn't been beaten, there'd basically been no progress in it which I thought was really fascinating given the fact that we've like scaled up these language model systems by almost like 50,000 times over the last,…”

“Pure LLM systems are scoring like zero percent now again on on arc B two single COT systems like R one and O one score like one percent.”

“I actually think we're going to start to see agents start to work this year specifically because of progress on arc.”

“I think anyone who's like Just doing model training at this point is like lighting money on fire. If you really want to make a unique difference, especially if you're a small startup, like a founder, like you gotta go take an orthogonal approach. You gotta try…”

“I generally think he's been more right than wrong. I think if you like just take a limited five year view on this from 20, 20 up until 20, 20, end of 20, 24, you know, I think it was a generally right. Like he was making the right ideas.”

“I actually don't think either is sufficient. I think some merger of the two is what's necessary to get to AGI.”

“Just sort of objectively, language models do not work to beat Arc. And people have tried.”

“I am more confident actually that or I guess I would bet that the solution arc probably comes from an outsider. I think it's probably gonna come from somebody who's sort of not indoctrinated in the current way of thinking about language models and scale.”

“We don't send any internal email.”

“We do think that efficiency is actually a really, really important aspect of intelligence. You know, you can brute force your way up to intelligence, but we really do want to be shooting for like human targets and efficiency for this stuff.”

“My hot take is, like, I guess, yes, you should still learn to code. Primarily because it's been, it gives you like leverage over technology today. And yeah, like I don't see that leverage over technology going away anytime soon, particularly if you want to wor…”

“You know, it fails randomly two out of 10 times, which might be fine in a supervised setting like ChatGPT. You know, where you're talking with these sort of assistants but it doesn't really work in an automation case where it's hands-off keyboard running on a …”

“We don't have AGI yet. We don't have the ideas for it yet. We're still our idea constrained even literally today.”

“Frontier AI research is also basically like completely stopped publishing. You know, the GPD four paper had zero technical details. The Gemini paper had zero technical details on the longer context stuff.”

“There's an AI lab called lab 42 out of Switzerland that's been running a small annual contest over the last four years to try and beat this eval and state of the art today is. 34% state of the art four years ago when it was first introduced was 20%. So we've m…”

“Francois definition, which is the one that I think is the right one is this definition that general intelligence is a system that can effectively, efficiently acquire new skill. That's it efficiently acquiring new skill and being able to solve these open-ended…”

“And I suspect now over the last four years, we have. We might now have enough compute that's come online at a cheap enough, like kind of cost per flap that some of those old neural architecture search methods we should revisit and relax the search.”

“I mean, I think my sort of underlying beliefs on AI are AI should likely get regulated through the existing regulatory frameworks that exist. I don't see a lot of new harm or use cases or damage caused by just the narrow form of AI systems that we have today t…”

“Your work can't be your family at Zapier or any distributed remote company. Like you, in the past, it's very easy to lean on your work as that, like, social connection.”

“The types of things you have to do in order to be a successful remote company make you just a generally better company. They are not unique to remote. However, they, you do have to figure them out earlier.”

“A secondary benefit that I think isn't as obvious is that when you're actually doing your job, like, the best work gets done not when you're, like, sitting next to someone and, like, collaborating all day. There's Like, you have to get into deep work. Even for…”

“Like, hey, I get the hype, but like, they just, they're not reliable enough yet. You know, they don't work two out of 10 times, and that just doesn't work for these unsupervised automation products.”

“And I think arc shows that like, yeah, there actually are frontier problems. That are unsolved, that individual people and individual teams can actually make a difference on today.”

“It was about a million bucks back in 2012. We never spent the money. By the time we actually got the round closed, figured out who we wanted to hire, got the payroll started, like revenue had caught up. And so literally I think you could trace every dollar we …”

“There's maybe a couple of million deep learning engineers now in the world. In contrast, there's probably maybe only a few hundred like really great program synthesis folks across, across the world.”

“Over fifty million AI tasks have run on the platform to date over the last year and a half or so since we started tracking.”

“Using AI in the middle of a workflow is, is kind of the dominant way people are adopting AI today.”

“Just because of the dynamics of the explosion and how many tools there are, there's no one player in that marketplace that's going to be incentivized to go build thousands of integrations with everyone else.”

“I think I'd always, I'd heard the heuristic of like once you get big enough where you could carve off one percent of your revenue and that could be its own standalone business, like you've kind of reached a critical mass that you could actually build a platfor…”

“The value we gave to partners was around retention. If they built, they integrated with Zapier, they got access to 50, 60 at the time integrations that were maintained and scaled and were adding more to it, and they got it left for free. So it allows them to g…”

“And it's like, well, yeah, we have, you know, a hundred, we have like fifty million in ARR, like, yeah, we're a real company.”

“If you, like, one of the things I tell a lot of our teammates, if you see that, that's probably a really good signal that you should be, like, jumping on a video call at that point. Instead of wasting, or not wasting, but instead of spending, you know, 10 man …”

“In Slack, if you tag somebody in a message, like, at tag them specifically, it's kind of the social norm to be, to acknowledge that within 24 hours. So we have some expectations like that, and the reason we have said 24 hours is because we have folks all acros…”

“If you're the type of person who like, as soon as I get blocked, I'm just going to sit here until I'm told what to do next. You're not going to be successful at Zapier or I argue most remote companies.”

“I like to think about there's like the things that are important to be consistent across teams are the interfaces. Like, you need to make sure that the interface between teams is consistent so that both teams know how are you.”

“And there's kind of a meet in the middle approach where you kind of want the work that's happening, 50% of it to be kind of top down driven, and I think 50% of it being bottom up. Because in reality, the exec team leadership is never going to have perfect insi…”

“And very early on, we set the expectation that Slack is not a tool you're expected to keep up with in Zapier. You are free to leave channels. In fact, we encourage it. Kevin Hale: That is fascinating. Mike Knoop: There's a feature in Slack where you can turn o…”

“Even though we're a hundred percent remote, there is still a lot of value for getting in person. and we don't like discount that. you get to build a lot of empathy. You get to build relationship with folks and it allows you to kind of be, assume best intent …”

“I think it is one of the reasons why it is so hard to add remote onto an existing company is because remote while When you see folks talk about it and ask questions about, it's always very process and tools based. I think the honest answer is that it's more of…”

“Opening eye situation was a little different because they had reached out to us and said, Hey, we think we've got a really impressive result on the public eval set. And we'd like your help to verify. On the semi-private set, which is what we created that data …”