AI agent

“Part of why I think it's early is because the issue with agents is it's not quite like the internet where you could like make a website and the website would appear. The issue with agents is that they don't work. And so it may be a bit early to figure out what…”

“Reasoning is actually, we believe the biggest blocker to agents or systems that can do these larger goals.”

“I think what we'll see is we'll get like relatively simplistic agents pretty soon, and they will get more and more complex. And there's like a future wave in which they are able to do these like really difficult, really long running tasks. And the blocker to t…”

“Chat as an interface is skeuomorphic. So in the early days, when we made word processors on our computers, they had notepad lines because that's what we understood you know, these like objects to be chat. Like texting someone is something we understand. So tex…”

“We're not going in the agent direction, right? I mean, I'll believe in agents when somebody shows me one that works.”

“Everyone under values the importance of really good sensors and actuators. And actually a lot of what's helped us get a lot of reliability is like a really strong focus on like, actually, why does the model not do this thing? And the non-trivial amount of time…”

“It's like on each stage of the node, you're gonna have to see a marginal improvement in the probability of success for that particular workload because of non-determinism.”

“Currently though, make a model into an agent by just copying and pasting error messages for the most part. And that's what I do is, you know, you run it and it gives you some code that doesn't work and either I'll fix the code or it will give me buggy code and…”

“And there's such a long tail of other ones, but in practice, like, when people go to production, they generally have their own tools, or maybe one of those three, maybe some other ones, but, like, very, very few other ones.”

“I think of it in terms of like jobs to be done, and I think of it in terms of who is the Lindy serving.”

“The more you can put your agent on rails, one, the more reliable it's going to be, obviously, but two, it's also going to be easier to use for the user, because you can really, as a user, you get, instead of just getting this, like, big, giant, intimidating te…”

“We actually very often talk about how the business of the future is like a game of factorial. It's like you just wake up in the morning and you've got your Lindy instance. It's like Slack and you've got like 5000 Lindys in the sidebar and your job is to someho…”

“Eventually, I think that that's where the world is going to. Like the code It's going to be there, and we're, there will be developers, et cetera, but as agent improves and capabilities of the LLMs and integrations to different parts of the environment, develo…”

“We're already developing things for programmers, you know, How is an agent different from a programmer? And it is different, obviously, you know, like agents are different from programmers, but they're not that different at this point, so we can kind of intera…”

“Right now, actually, agents are somewhat expensive to run in some cases, but I expect that that won't last six months. I bet we'll have much better agent models in six months.”

“And I think the skill floor more and more, I think, 20, 25 will be the first year where the AI sets the skill floor of a role.”

“Sometimes I feel like a lot of researchers or, like, people in the AI community are, like, so into, like, yeah, agents, delegate everything, like, blah, blah. But, like, on the way towards that, I think, like, collaboration is actually one of the main roadbloc…”

“Prompts are great, but it's not actually a complete specification for anything. It never can be.”

“I actually think for every single agentic domain, whether it's customer service or legal or software engineering, that's essentially what the company building those agents is building is like the system through which you express the behaviors you want, esoteri…”

“To go use that metaphor, we're sort of in the jQuery era of agents, not the React era”

“And actually, well, I'm sure we'll get into AI, but I sort of feel like we'll go through that evolution with AI agents as well that I feel like we're missing a lot of the core abstractions that I think in 10 years we'll be like, gosh, how'd you make agents bef…”

“I have an intuition that agents will speak to agents using language for a while. I don't know if that's true. But there's a lot of reasons why there, that may be true. And so, you know, when your personal agent speaks to a Sierra agent to help figure out why y…”

“If you build, like, five, six minute jobs, they're bound to be, like, failures and you don't want to, like, retry, lose your progress and so on, so this notion of, like, keeping state knowing what to retry and kind of keep the journey going.”

“I actually think that off is the biggest thing that will prevent agents from accessing stuff, not captures.”

“I think there'll be agent off in the future. I don't know if it's going to happen from an individual company, but actually authentication providers that have a You know, hidden login as agent feature, which will then you put in your email. You'll get a push no…”

“And hopefully find the right answer and then say, okay, this was actually the right one and memorize that and go there in the future on the roadmap for sure.”

“I think like one thing you see a lot when you talk to people building agents is there's like some effective context length that actually like has the model be the smartest. And that seems to vary slightly model by model, but for this model, for whatever purpos…”

“I think this problem is gonna have to be solved for agents to take off, quite honestly. I mean, I see what you're saying, but we do have to at some point then pass identity onto, like, if we go back to what we were talking about of, you know, I would love an a…”

“An agent is really like LLMs and a bunch of workflows and coordination and orchestration and then extra, like some sort of services, right? Like you have kind of the brain of the operation, which is the LLM and it can come up with a plan. And then, but then it…”

“Especially with people building agents, I think it's kind of funny because it's a bit, to me, the anthropomorphification of agents, where it's like, we humans use browsers, so my agent is going to use a browser, but eventually I think that will get replaced wi…”

“I think that we're going to see a wave of, you know, going back to, again, Sunil, your point about mom and pop shops, these businesses turn up that are agent first, and that's kind of the only interface to using them as opposed to, you know, UIs and APIs and o…”

“I think the next challenge is observability and okay, I've gone and spun off. I have 100,000, maybe millions of agents that are running. How do I track their different states? How do I track where their stock? I think that that's a really interesting next prob…”

“These little instances of compute that you can write a code that you can spin up really quickly and scales horizontally across the planet is the perfect little container for running your agents in.”

“If we want to do something even more meaningful, it felt like the next kind of advancement is not this kind of, I'm chatting with some software in a kind of a synchronous back and forth model is that software is going to do things for me. In kind of multi-step…”

“So we're used to the chat bot back and forth. Fine. I get that. I think we're gonna move to a blend of some of those things are gonna be synchronous as they are now, but some are gonna be async. It's just gonna put it in a queue”

“So I think it is, I will go so far as to say it's inevitable that we're going to have hybrid teams someday. And what I mean by hybrid teams. So back in the day, hybrid teams were, oh, well, you have some full-time employees and some contractors. Then it was li…”

“The last thing I can mention is we use contractors for evaluation where we cannot Do automatic evaluation. So with chats and agents, it becomes way harder to do things automatically. And so we use contractors for that.”

“The first thing we always do is, like, try to see if you can build, like, regular automation, see that scales, and then build the agent that, like, supports everything. What I mean by that is, like We don't want to over invest in trying to get an agent stood u…”

“Yeah, I think it's a good idea to stop thinking about a sentence was just for code interpreting. And more about like a runtime, code runtime for the LLM, or the agent. The use case for the sandbox, it's a very horizontal in a sense that it can cover everything…”

“So we have these kind of two reward signals that compliment each other to and the reason why this is necessary is to avoid certain, I guess, behavioral collapses where a model might choose, for example, to mine coal and mine a billion or a trillion coal. And t…”

“We wanted to collaborate, not commit by commit, but keystroke by keystroke. And that's kind of a weird thing for human beings to want to do. It seems like in this world, like our team works that way. We work a lot together in real time, but a lot of people, th…”

“A lot of, like, agents that I see are really impressive, but it's basically, like, part of what's impressive is it's like a bunch of developers building this, like, really bespoke state machine around a bunch of, like, short model calls, and so then the upper …”

“The promise of test driven development is going to finally be delivered with this world of AI agents that are working on software development”

“The outer loop of software development and what a software developer does, planning, talking with other human beings, interacting around what needs to get done, is something that's going to continue to be very human-driven, while the inner loop, the actual exe…”

“I do see these agents generating spreadsheets to answer people's questions, particularly when they focus around the user understanding data.”

“We do a little bit of vision when users are creating charts. We pass back an image of what the chart looks like to the AI, because a lot of times the user will describe visually the change that they want to the chart. They'll say something like, this doesn't l…”

“And so if you have an AI model that's, like, actually really aligned, To you and your preferences, then that could end up doing a way better job than a human could. Well, not, not that it's doing a better job than a human could, but like it's doing a better jo…”

“So we'll show the same thing to AI, except that AI is gonna get a JSON grid list of lists. So those get a bunch of numbers, 64 by 64, and they can choose to turn that into an image if they want to, or agnostic, do whatever you want with it if they want to do m…”

“What you see in general is that that doubling time is about every seven months, which already is pretty crazy, actually, but in code, it's actually even faster. It's every 70 days, which is two or three months, and so, you know, if you look at various software…”

“So we very strongly believe in a future of no copy-paste. Right? You should never have a situation where you're in a terminal, or you're in a document, or even on a website, and you're copy-pasting text into an agent. That's just not how the way the world work…”

“Copilots are collaborative, but can only work on small scopes. Agents can do larger tasks, but are not collaborative. Both are very useful, but we realize that this real magic will happen when the AI has the ability to be both collaborative and independently p…”

“We are kind of self evolving skills now for agents. So people building or using agents want to like basically make Agents interact with their apps. They can use Composio. We manage all the authentication and user account related stuff for them so that they don…”

“And then you have your codex infrastructure that has a local agent and a remote agent, and that is able to seamlessly, you know, interplay between the two and then is able to multiplayer. Like, this is what the future is going to look like, and it's going to b…”

“Because you tell an agent, it can't run a command. It's going to write a script to run the command.”

“If you're working at Any decent sized shop, let alone an enterprise, the way you set up your dev environment is really hot at the heart of all of this. And of course you want your agents to work in exactly the same space that you as a human can operate in.”

“Rather than just writing back the full context of your tool calls, which could be token heavy, write those to disk and you can write back a summary. It could be a URL, something so that the agent knows it's retrieved a thing. It can fetch that on demand, but y…”

“When you kind of put together an agent, it's just tool clawing a loop. It's relatively simple to lay out, but it's actually quite tricky to get it to work well. In a particular, managing context with agents is a hard problem.”

“What patterns really worked well in self-driving cars, which is weirdly a very similar system to, you know, agents of today where you have like these cascading kind of like units that are all machine learning based and, you know, they all kind of like feed int…”

“What we're seeing now with agents is as soon as somebody has seen what it can do, they have such a multiplying effect or this brings so much value that people are willing to adopt the code base for this. Like the first time in how many decades where people are…”

“When you spin off dozens of agents, the sort of attack surface of the of the solution expands geometrically.”

“I also think that it's going to be something that as we move forward in time with more Asians writing more parts of your code base, you will also be less familiar with the code. And so then you might want a different abstraction where you're able to work on th…”

“I think that the chances that like everyone should be, or, you know, everyone who's deploying an agent at scale should be doing RL with it, either as part of sort of like a, you know, like pre-deployment or even like continuously as it's deployed, that that's …”

“I think that there is today, like. 10 times as much AI inference that could exist than is existing right now, just Purely with projects that are like sitting in the proof of concept stage and have not been deployed because there's like huge bucket of those. An…”

“If you're just purely training on kind of like an LLM user simulator, it's going to have its own idea of, like, what the correct way to answer is, and the breadth of, like, a way a human might respond in this situation is wider, and your agent just may not be …”

“For RL to work, you have to be looking at real runs, ideally of your actual agent in its current state across within an environment as real as possible.”

“I don't think the future of all agents is people NPM installing them onto their computers and then typing commands into their terminal. But I do think that behind the scenes, these agents are running in containers and their tools are actually programs that the…”

“Now you can think about, okay, actually foundation model, text alignment, like human readable description of like why we're worried about this charge. And then today, a human tomorrow, an agent sitting on top of that and decisioning, like reasoning over The mo…”

“At this moment, they're, like, agents are both extraordinarily effective and still very ineffective. And you have to find the right problems. And then when you find the right problems, they are super magical. And if you wander beyond, then they don't work.”

“Basically where the idea is that you go around your company and you ask people like, what do you hate most about your job? And I really think it finds the sweet spot because it finds problems that are, they're boring because they're tedious and repetitive, but…”

“We basically sign contracts with companies saying, okay, you have to commit to building three agents, and if you do, we're going to help you, like, we're going to build the first one for you. And then the second one, we are going to be there essentially by you…”

“Agents will really only work where we get to not only like the more intelligent models, but better reliability of the infrastructure providers, right?”

“I think a lot of success depends how well you flesh out what you wanna do, right? Like if you just use this and ask it to like start here executing a plan, it's gonna be high variability in what you're gonna get back, right? But the more we go through here and…”

“And I don't need, typically, the agent to say, hey, I found this lot, and this lot, which one do you prefer? I just asked for 15 minutes. Find it. Do it. I have an admin, when I was asking her, like, on Slack, find me 15 minutes. She's not asking me if I need,…”

“And I can continue just by sort of like saying that that's definitely going to be something I think is going to be something that we'll be capable of in 20, 26.”

“But if the client and the server don't know each other, we just don't have a good solution for now.”

“OAuth itself is for the most part, a very human centric protocol. It's just, it just tells you how you obtain a token. If you don't have a token, once you have a token, actually it doesn't matter.”

“I think that could change, you know, like, as agents actually become kind of, like, more prevalent and are interfacing with each other, and therefore, like, perhaps, like, the number of transactions explodes.”

“And I was surprised to hear the consensus was that most people there were using agents to collaborate on like building a design document and like, Collaborating on the architecture of the solution that they want to build, and then they'd be asking it to like e…”

“We don't actually have to hold their hands so much anymore, or like they don't actually need to necessarily have an automated lab. They can like write an email to a CRO or something, or they can like tell you what experiment to do. And you can take a video of …”

“When you think deeply about like what is missing, you kind of realize like, you know, that they still can't reliably do enterprise work, some of these agents. And so, you know, we started thinking about it and like it kind of struck us that You know, one of th…”

“If you didn't pay any like human cognition to get there, I don't think you're going to be a great reviewer. One of the reasons why, you know, what makes that, that human feet, that loop well is because once upon a time you did that and you could make the three…”

“We've hit some capability that you can do, you can build these much bigger blocks now. And those bigger blocks are not just like the single line of code. It might actually be a business.”

“The state of people making claims on agent performance is very unscientific and much more anecdotal and sometimes influenced by marketing desires.”

“So far we've been in, in easy mode. We've hit the easy button with AI, which is the agent just is you. And when you're in cloud code and you're in cursor and you're in codex, you're just, the agent is you, you're offing into your services. It can do everything…”

“DevRel is hot, like the hottest thing of all time right now. I like, if you could produce a fricking factory of DevRel people, like there's just like unlimited jobs right now on the other end of that. Cause we're gonna, everybody needs their Services and APIs …”

“But no matter what, there's going to be 10 to a hundred times more code. So I think you can be very long engineering right now as just a, you know, purely on the dimension of software is going to become increasingly more important. Once agents are, you know, t…”

“What's happening is we are changing our work to make the agents effective in that model. The agent didn't really adapt to how we work. We basically adapted to how the agent works. All of the economy has to go through that exact same evolution. The rest of the …”

“You know, whether you think the number is 10 X or a hundred X or whatever the number is, we're going to have some order of magnitude more agents than people. That's inevitable. It has to happen.”

“The person who creates the agent probably is gonna, for the foreseeable future, take on a lot of the liability of what that agent does. That agent doesn't deserve any privacy because it's, you know, it can't fully be autonomously operated and it doesn't have a…”

“If a really, really smart human could not do that task in five or 10 minutes for a search retrieval type task, you know, your agent's not gonna be able to do it any better.”

“Agents will certainly cause us to be much better organized around how we work with our information simply because the severity of the agent pulling the wrong data will be too high. And the productivity gain of that you'll miss out on by not doing this will be …”

“And then they all are having to understand that now you've got this new customer, which is the agent. And they've been building for two types of customers in the past. They've been building for users and they've been building for like applications. And now you…”

“All of that information becomes valuable to the enterprise and it's going to become extremely valuable to end users because now they can have agents go find what they're looking for and produce new New value and new data on that information. And it's going to …”

“I always laugh when, you know, people say, you don't need to be an engineer. Don't do computer science. I actually think like that is like still one of the most protected job categories because things are only getting more technical. Things are only going to g…”

“There's a part of the company that is sort of do or die for the agent wave. And it only happens to be more of my focus simply because it's existential that we get it right.”

“And that judgment is like a really new thing that the model needs to be able to have is like, when should it give up on a task? Cause, cause you just don't, it's a, can't find the thing. That's the real world of knowledge work problems. And this is the stuff t…”

“I think there's a difference between, like, people who want to build, like, Reliable software and people who want to build a cool demo. I think that's the core. The incentives are different. Yeah. The incentives are of like, okay, if this is right, 80% of the …”

“We will see before the end of the year an agent that is capable of running for longer than 24 hours with like self consistency the entire time.”

“I was chatting with folks at Stripe last week and saying, hey, you got to make the Stripe CLI actually tell agents what they can do on Stripe because that way they're going to use more stuff on Stripe. I think this is a real trend for the entire industry.”

“Dreamer is a new product which everyone can come and play with today. It's a place where everyone, literally everyone, can discover, build, and enjoy and use AI agents and agentic apps. And we really did design it for consumers, for folks who are not necessari…”

“We've actually find that making things show up in the other apps that you already use in your life is incredibly powerful.”

“So it turns out what we now know is an agent is the following. It's a language model. And then above that, it's a bash. It's a bash shell. So it's a Unix shell. And then the agent has access to the shell and, you know, hopefully in a sandbox, maybe in a sandbo…”

“The agent hiring the people, which of course is going to happen, right? It's obviously going to happen.”

“The CLIs are nice because they're super token efficient, and they can be made more token efficient really easily, right? Like, I'm sure you all have seen, like, I go to Buildkite or Jenkins, and I could just get this massive wall of build output. And in order …”

“A PR comment, a failed build. These are all signals that mean at some point the agent was missing context. We've got to figure out how to slurp it up and put it back in the reboot.”

“If we want to actually, like, make it see the layout, it's almost easier to rasterize that image to ASCII arc and feed it in to the agent.”

“Fundamentally, the agents are good at following instructions, so give them instructions, right? And it will, you know, improve the reliability of the result, right? Like we, much like the way we use Symphony, we don't want folks to have to monitor the agent as…”

“The ability to do, like, any simple find-and-grab information from a dashboard, like, I think that just has to be agent-compatible, like, this year for companies, because that is one of those things where if you spend 10 minutes searching for a key that you ca…”

“One of the things that people quietly talk about is that that's very fatiguing. And I know some people who are operating that way. I've met these people and it is fatiguing. And what is fatigue really going to hit you on? Fatigue is really gonna hit you when n…”

“And actually, 99% of the time, it's a bug in one of the tools. Right. And so, just fix the bug.”

“And then I think the most important thing that's super cool is that there it's also inherently a bootstrapped. So if there's an issue of the agent can debug and fix itself within the same environment that it uses the tool, right?”

“There's a critical difference To being a manager, which is that like, it is actually very deeply technical. The problem of, you know, humans are very like fuzzy and you can't like treat a team of humans like a rigorous system where like, you know, PRs like fle…”

“And if generally we're in the kind of space right now of trying to shift decision-making from, you know, kind of human brains into agentics brains, right? It used to be from wetware to software. I don't even know what to call kind of the LLMs now, right? Into …”

“So my view on it is that in my head, it completed the quadrant of the types of data sources that are required to reach what I've been talking about as kind of escape velocity for agents in production. And so what I mean by that is I think it's exactly four ty…”

“Nowadays though we expose all of the underlying APIs for that. And now using AI agents, you can actually configure a sensor suite with just text and likely reach a better result than you could have through the GUI in the past. And we're taking that thinking no…”

“Almost every agent is a coding agent underneath underneath the hood, right? So you give it whatever, a file system, it can write its own code and so forth.”

“So all these things that we've built for, I actually think the things we've built for humans are actually the things that are going to be continually durable.”

“Things that were prohibitively annoying to humans are not actually prohibitively annoying to agents. They're really, really nice, right? And so, if I wanted to hand you a CLI and I said, hey, guess what? The CLI has 40 arguments and. Right. 600 flags. You'd be…”

“We have previously used it a lot as an input and its goal moving forward is actually a lot more like an output. What I mean by that is you would go to the canvas and you'd make some changes and all these other things, whatever. Right. And you see them and, you…”

“Most people thought it was the same infrastructure for humans and agents. We understood a quarter ago. It's not, we just didn't know what was the right primitive.”

“Agents will be like humans in the sense of you don't want your laptop to be shut down until you're done with work. Like, and you want to close the lid and open the lid. It's the same state. So agents would want that like pause and come back. They want those tw…”

“There will be a cloud built out specifically for agents. And so that cloud will have sandboxes and it will have web search and it'll have databases like SQLite or Neon or whatever, specifically for agent and other things. We are not at the end of the new infra…”

“Last week, they released a new system for agent evaluation. It's like a very, like, experimental initial benchmark, but pretty much allowing agents to take an exam and compete in a leaderboard, which is always fun.”

“I think the reason why there's not a single answer is ultimately we're trying to codify trust. We're trying to say like, okay, if Sean reviews this, I'm going to trust it because you're Sean or you're the senior dev or you're the whatever. And right now when w…”

“And so I do think there's something true there where having like either taking only what you need or the dependencies just getting incredibly small over time, I think will help to some degree, but it's not going to solve the fundamental problem. I don't think …”

“Operating systems need to look different than they looked five years ago because it's not just you using them anymore.”

“One of the problems that we have, right. Is that our agents, if you install them, not on a Mac mini or not on a hosted device, you install them on a personal device or a work device. We need better sandboxing at the OS level.”

“The agent bought like a shit ton of tomatoes two weeks earlier. And before the opening and now they're all rotten.”

“I think it can be done today, but you would do it in like e-commerce where it's like the probability of success is like really low, no matter if a human or an agent does it, but like an agent could surely manage everything.”

“So I think this is really sort of an underappreciated point that we're reaching this point, this sort of phase where a lot of security, a lot of science has this potential to kind of explode. Not because we're going to get better at it, but because agents can …”

“So far, we are still a lot, in a lot of cases, operating on the condition that your agent has your permissions. Yeah. That is a very standard default. And I think that will be changed. I mean, your permissions may be in a sandbox, but still kind of your permis…”

“I think in terms of how this will evolve, actually, I don't think it'll be per app, but I think what will happen first is people have different personas that they have, right? So you don't want your work life and your home email to be mixed up. Yeah. Right. A …”

“For this type of interoperability where multiple parties that are moving at different speeds are building stuff and you still want some layer on top to coordinate you do want to design it and build it. So it reminds me of that, like agents talking to each othe…”

“That's where we've seen a lot of other agents like hit things like people think they prototyped an awesome agent, but you know, it's not allowed to connect to like some really important data or whatever because of the, Security team.”

“There's no retrieval-based search, semantic search, keyword matching search that's going to be able to give your agent that information.”

“Agents are different than normal software in that normal enterprise software. You can deploy it, put it on some server and let it run for like five years. And as long as you never touch it just works. Agents don't really work that way. The core thing that you'…”

“Sometimes it makes sense, like, if they're reaching for this thing, it's product feedback, like, give it to them.”

“I think the things that sometimes agents struggle with without right guidance and a skill is how to use the rest of our observability. Like, how to, something is failing, like, How do you look at the logs and then update the right thing? It's sort of reasoning…”

“So ultimately reliability has less to do with clicking in the same place and scrolling and more to do with modeling the user's mind. And that shift is everything that reframes how we think about what it is that we're building.”

“We look at what the metrics of the actual agents and they're so unreliable that ironically we feel a little bit better. The AI is actually not where we need it to be. To automate enough of the work.”

“So now like we're seeing with agents, like there is Probably a contingent of like early adopters still who, you know, truly get it. We're like, you know, you can do anything. You just have to make sure the right context is there. It's connected to the right to…”

“So the way that we're going to need to reinvent what that kind of interactive interruptible work looks like in the agent layer, which is separate from the work going on, I think in the, like in the model layer. And in this like interactive model space, but I t…”

“This technology around simulation, creating accurate representation of people ought to precede the more complex agents that would automate the world that we live in.”

“If you're running your model at over 4000 tokens per second. Now the, you can do, you know, more agentic loops. You can do more reasoning. Ultimately you get significantly more capable, more intelligent agents.”

“It's moving way faster than any other technology adoption that I've ever seen.”

“Really an agent is the ultimate settings screen for any software and code is the ultimate setting screen for any software.”

“And compared to a few years ago, You do not have the need to go and hire people that you do not trust, that you do not trust to have high agency or the right skills, because you can use an agent to do those things.”