The Exchanges

Q incredible amount of access to GPUs and data, but for context engineering, everyone here could do it. You have, you just need the API to something like Gemini, and then work on your own setup for your own retrieval, your own tool calls, and et cetera, et cetera. So, How does, what are some tips for everyone here? How does everyone get better at and become exceptional at context engineering?

A Yeah, I mean, I think, uh, a really good way to do it is to use these models and, and sort of harnesses and tools and so on to try to solve problems. And then some, sometimes you can actually see where the models are failing. And often you can actually make the model work better and succeed at that kind of problem by not just adjusting the model parameters, which is hard to do from the outside, but from, you know, creating better guidelines for the model, you know, writing skills for the model to know how to use different tools that would be incredibly useful for solving this particular class of problem. And I think as you do that, you end up on this kind of Improving, self-improving of the setup that you're trying to use to, to solve things. Uh, and you know, that, that's a really good way to get better at understanding what, what additional information the model would want in order to become more capable.

AI assessment note: “use these models and, and sort of harnesses and tools and so on to try to solve problems”

Q How do you find those? I mean, are those things effectively, uh, out of distribution from the training set, and what exactly is the problem shape that fits that?

A Yeah, I mean, I think, uh, sometimes it's Uh, a product that you build that might have access to particular kind of data that the underlying model might not, the general model. So it might be you're building something to help users organize all their own personal information, and the model won't necessarily have access to that. And so there you can have a big advantage because all of a sudden your model has visibility or your product has visibility into important data. Um, It could be some incredibly hard problem where if you get the right training data and you can train a more specific model than a general purpose one, you can actually do that in a very affordable way. Maybe it doesn't take that much compute to train a niche model for this particular problem, but you can get something that's highly accurate. That can sometimes be a really good building block for, for solving a important problem that is maybe not Handled very well by the general model.

AI assessment note: “a product that you build that might have access to particular kind of data”

Q How are some ways you implemented this particular workflow for your agents internally?

A Yeah, I mean, we have, ah, you know, harnesses, and then we have a whole set of skills, ah, particularly in the internal Google development environment, we have skills so that the agents can know how to use lots of our internal tooling for coding, or for code reviews, or for, you know, measuring performance, or, you know, fetching log files, and, um, those are just skills that you can add to make the base model more capable, even though it hasn't necessarily been trained on exactly the way that You know, Google internal, ah, engineers would fetch log files from our, you know, proprietary system with the right kind of skill, ah, definition, you can actually get it to work. And that, that improves the usefulness of the agents.

AI assessment note: “we have harnesses, and then we have a whole set of skills”

Q Now let's assume now every founder gets good at running hundreds of agents at the same time, and all the code is written for them by the agents. What becomes the scare skill?

A Yeah, I mean, I think it's really having incredibly good taste in what you ask your agents to work on, right? That is the, the crux of, you know, from my background, ah, a research problem. You know, a researcher can have all the tools and all the techniques, but often most of the battle is what problem are you gonna spend your time on? And if you pick the problem well and you succeed in, in, in solving it, That's way better than if you, you know, ah, delightfully execute a research investigation into a rather boring problem. And so that high level wisdom of what to work on, I think is incredibly important. I think models are not necessarily going to be that good at it. So you're going to have people steering Uh, a lot of AI assisted computation in order to accomplish great things and more quickly. Um, but that essence of, of what it is you want your models to do is the, the key thing you should focus on.

AI assessment note: “having incredibly good taste in what you ask your agents to work on”

Q a bit about more about that second path of working with people that you really like in a small team. You've been able to be an incredible mentor and manager to many, many engineers, and you've been able to build huge systems. Um, what are some, some of the lessons for everyone here on how to get the most and how to work with smart people or find smart people?

A Yeah. I mean, You always want to find people who have really good skills in some, some area that's needed in, you know, a team you're trying to form, whether that's inside a company or starting a company. Um, but you also want to find people that are People you delight being around, right? Because you're going to spend a lot of time around people working on really hard problems, and you want people who are low ego, that are team players, that, you know, have complementary skills to your own, perhaps. Um, I always find working in a small team where people know things that I don't know, and where maybe I have some skills that other people don't have as much of, You know, it's super fun because you're collectively building something or working on something that none of you could maybe do individually, but in the process of working on that, you actually gain a lot of new knowledge and new skills, ah, for yourself, and so do they. And you, you kind of want to view your engineering or research career as you have an amazing tool belt of techniques, and you always want to be adding new tools. To that tool belt, because you never know when you might come across a problem where you need these four specialized tools rather than these three. And adding more tools makes it more likely that the problems you, you encounter in the future will be solvable by you.

AI assessment note: “You always want to find people who have really good skills in some, some area”

Q Incredible napkin math. Napkins are good. Napkins are good. So actually, what's a good napkin math that everyone here who wants to be a future founder should run tonight to potentially build something as consequential as the TPU?

A Yeah, I mean, uh, it's always hard to say. Um, I think, uh, Think about what problems you see and whatever it is you're thinking about, what, what bottlenecks you see, and are there very different ways of thinking of the solutions to some of those problems that would get you, you know, an order of magnitude or two orders of magnitude better, uh, performance or capability or whatever it is, um, you know, because sometimes if you just squint at a problem and you think about not necessarily being anchored on exactly how that problem is solved today, But how you would solve it from first principles, you can come up with really good ideas that are, you know, maybe not what other people are thinking about.

AI assessment note: “Think about what problems you see and whatever it is you're thinking about”