The Exchanges, every show

Q Final one for you. What are you most excited by?

A We talked about Jeff Dean. I'm, Jeff Dean? Yeah. Uh, I'm most excited by applications of AI. To pharmacology and biology. So, uh, I have a family member who has very severe autoimmune neural inflammation. He's had it for six years. I took a blood sample from him every week for 15 weeks, sent it to a lab, sequenced his genome, uh, did proteomics on it to figure out how the proteins are expressing in his body, and run an RNA analysis in each one of those weeks, and then I compared that to self-reporting data on what the quality of life is and what his mood effect was, Uh, every day I have, like, six years worth of data on him, and ran it on a GPU cluster, and, like, I found so many things that no doctor could ever tell me, and he has a very rare disease. It was called an, like, an orphan disease, because there's not that many people. There's a Facebook group for this disease. I'm buying now, uh, basically, like, you know, it's like a 5000 dollar device you can fit in your pocket, but if you put a piece of hair or saliva or blood into it, it can sequence your entire genome, And so I'm organizing meetups with all the people who have this disease to sequence all of their genomes and then compare them all on a gigantic GPU cluster to figure out what epigenetic common thread there is between them. And I am, I know I'm going to solve this disease. I would have never had an edge to do t…

AI assessment note: “I'm most excited by applications of AI. To pharmacology and biology.”

Q What was your biggest lesson from working with him? If there was one takeaway?

A That you can overestimate what you can achieve in one year and underestimate what you can achieve in five. When I started, it was a three-year-old startup, and it was basically a glorified contact manager, you know, Salesforce automation. Um, we essentially said what you're traditionally using ACT or Goldmine or using a spreadsheet, you can use Salesforce for. But Mark had this bigger vision and he said, we're going after Siebel, SAP, Oracle, Microsoft. We didn't have the product set to go after those competitors, but he was such an incredible marketer that he created this perception in the industry that some of the largest companies in the world could adopt this technology and that we would deliver on a roadmap that would satisfy the requirements over time. And he did.

AI assessment note: “That you can overestimate what you can achieve in one year and underestimate”

Q Why football? Why English Premier League? You can sponsor F-One, you can sponsor, you know, LaGuardia Golf as well. Why football?

A Well, I love the sport, but let's put that to the side for a minute. I think the value of these sponsorships obviously come in two forms. The first is awareness. And as we saw last night against Chelsea, this is a game that's being televised globally. So you've got millions of viewers that are looking at your brand and you're getting impressions, obviously. The second, which is obviously easier to quantify, is hospitality. And we're sitting here at Craven Cottage along the Thames. I think this is arguably the best sports experience in the world, and I've been to many. We had 20 executives last night attend an intimate Michelin grade dinner. We had, you know, the C-level executive come from Paris, one of the largest banks in Europe, just to experience that. And those types of relationships are extremely important, especially as we move up market.

AI assessment note: “I think the value of these sponsorships obviously come in two forms.”

Q they're about tenders. We see them more and more for employees, and I think talent acquisition is one of the hardest things today. I actually got in a lot of trouble the other day for this. I said, if you are trying to hire A-star talent today, you can't. OpenAI and Anthropics simply pay, and they go to the front-end model providers. Is that true, or was I being glib?

A I think it's true depending on the category that you're in. I think if you're a Digital native AI startup in San Francisco. It's a very difficult employment environment because you're competing against OpenAI and Anthropic and others, um, that are extremely well capitalized and are putting offers that are extraordinarily aggressive into the market. I think if you're a infrastructure provider like ClickHouse, uh, we look for a slightly different, um, profile. You know, we're looking for database engineers, people that have experience with distributed systems. Um, slightly different than what the frontier labs are hiring for. We employ people in 27 different countries, um, which gives us a competitive advantage, so I can hire engineers in Portugal and Germany and Singapore. Um, you know, we've got single digit attrition, so we've got extraordinarily high retention. Um, we have done some structured secondaries, um, and we'll continue to do so over time, but not with the frequency that I think some of the younger companies are doing.

AI assessment note: “I think it's true depending on the category that you're in.”

Q If you could have your way, would you not have everyone be in office in some way?

A I, I wouldn't, and I'll explain why. Uh, we're a very international company, uh, by almost every measure. Over half of our revenue comes from outside of the US. 40% here in EMEA, 10% in Asia. Over half of our customers are outside of North America. And so we need to support our customers in a variety of different languages, in a variety of different time zones. I mentioned we're live in 36 different regions around the world across all three hyperscalers. There's no way that you can centrally manage that. From one location. You need to have people in every single time zone. You need to have relationships with the hyperscalers in region. We go to market with AWS. We go to market with Google Cloud. We go to market with Azure. I flew to China to launch a partnership with Alibaba. Like you're going to have, you're going to need local language speakers to maintain those partnerships. And you can't do it from one or two or three centralized hubs.

AI assessment note: “I, I wouldn't, and I'll explain why.”

Q What am I missing? Is that, what should I know that I'm not getting? How should I think about that?

A No, I mean, I think that is fundamentally the risk for an investor is that you're making a two trillion dollar bet on one of the most competitive application markets in one of the most finicky segments of the market, which is, you know, dev tools. And so I do think that part of what needs to happen in order to make these Uh, companies like Anthropica and OpenAI realize the value is they either A, which they're pursuing, have to go through regulatory capture, in which case they go and they tell, they sort of scare politicians into thinking that they must own the means of intelligence, and thus they become the only providers of the most frontier capabilities, or B, they have to figure out a way to build applications and outcomes That match the true Pareto frontier of cost and quality. And I think that means opening up to more models. So it's actually like sort of at odds with the two strategies. You either, you know, capture it and keep the model or open up to everybody. This is a very hard decision and one that I think you can start to see OpenAI actually grappling with as they've let more models into their harness. They're not making it official, but they're Clearly supporting an open model ecosystem in a more direct way.

AI assessment note: “I think that is fundamentally the risk for an investor”

Q lot of guests on the show before have made Kind of bold statements that, like, 70, 80% of the neolabs that we have today will die in a given time period, three to five years, whatever you want to choose. Um, do you think that's true, and how would you advise me and other investors on the model or the neolabs that will thrive versus die in this next wave?

A I think it's plausible it's even more. I think it could be 80 to 90% of Neolabs die in the next 18 months. And die is going to be a funny word to use because it'll probably be for a lot of them incredible outcomes. So I don't know if it's necessarily doom and gloom as much as it's these businesses may not make sense as independent businesses. And so a lot of what I think matters for a Neolab is you should ask questions like one, Is this business attached to a durable workflow? Two, is that workflow going to change if new frontier models get better? And three, if this workflow were to be introduced to a new business, then would that new business figure out something even better? And so basically, is it durable to, like, effectively an entirely new way of thinking or a new way of working? If all three of those are true, legal is a great place where I think, one, New models won't necessarily get better without access to the data. Two, it's obviously a very proprietary workflow. And three, we're still gonna have legal system in five, 10, 20 years. So, probably all the neolabs focused on legal are gonna have great outcomes. Versus, I would argue that there's some places like, um, a lot of knowledge work that's related to intermediate tasks, like people operating in Excel and JIRA. That's just not gonna be differentiated. The workflows are very common. And I think that we may not use…

AI assessment note: “I think it could be 80 to 90% of Neolabs die in the next 18 months.”

Q let's, I think we should start by giving diesel generators their flowers. There's a reason why we use them everywhere all over the world. So we should talk about what's good and uniquely good about them, and then we can talk about what's bad, which in my opinion is a longer list. But first, like, what is good about diesel generators? Why are they fit for purpose for these applications?

A Yeah, there is a very good reason that they are so prevalent and nothing has been able to displace them so far, because it's a very unique combination of capabilities. So, number one, a diesel generator is essentially a highly convenient microgrid in a box. When you have the generator, you need nothing else to provide electricity wherever you are if the grid is down or if the grid's not available. It's also incredibly low cost. Diesel generators are on the order of hardware alone, five to 800 dollars per kilowatt. Even the more expensive emissions controlled ones are about a thousand dollars per kilowatt. You're talking about incredibly low cost compared to a gas turbine or supplicating engine. Or a fuel cell or any other, most other kinds of technologies. And then fundamentally, it's also about the fuel itself. When you think about what diesel is as a liquid fuel, it's a way to store energy incredibly compactly. Ultra high energy density. It's reasonably stable. It won't just, you know, catch on fire on its own spontaneously. It doesn't leak out. You don't need specialized tanks like you do with natural gas or other gaseous fuels like hydrogen. Um, so it's a very simple fuel Easy to store on site. You have a simple plastic tank. You can store 48 hours of fuel or 96 hours of fuel. What people in the battery world can consider very long durations is basically trivial to store wi…

AI assessment note: “number one, a diesel generator is essentially a highly convenient microgrid in a box.”

Q Um, let's talk about end life. So jumping in, give us the sort of overview. What exactly is the problem that you're trying to solve, and how are you solving it?

A The world does not have a copper discovery problem. We have a copper recovery problem. So the fact is we need so much more copper in the next 2030, 40 years, and we found a lot of this copper sometimes decades ago, sometimes hundreds of years ago, but Current technology can't get it out economically. It's very complex. There's low permeability. We can talk about how heap leaches work, but new mines take 10 to 15 years to permit and build, and those clocks between needing the copper today and new mines, they just, they do not align. The answer does not work there. So at Endolith, we use microbes and data to pull more copper out of material that's already been mined at operations that already exist. The microbes aren't new. They've been breaking down rock for billions of years, and this enables us to really equip and turn these heap leeches into a precision tool to get more and more copper out and bring it to market today.

AI assessment note: “we use microbes and data to pull more copper out of material that's already”

Q Right. And so I think this leads to you essentially answering this question, but, but I guess I was getting at kind of, like, why haven't they done some of these things on their own? Like, are they using any microbial approaches? Is there a reason that they That they shy away from this kind of innovation? Like, why?

A Yeah, so there's two parts there, and it's, it's partially because the downside is asymmetric, and honestly, if I ran a mine, I would be worried about that, too. These heap leaches that we're talking about, they are multi-billion dollar assets that have to run every day, so if an external party's innovation adds two points of recovery, that's nice, but if it disrupts the heap, Someone's quarter is destroyed. Someone's career might go with it because they're not hitting their milestones, and so the internal incentive structure for major rewards is typically saying no to startups and thinking about building internal versions which may or may not work. Um, I don't fight that logic. Like, that's not something that we're coming up here to disrupt, but we really want to work with it and design around it. So everything we do is really structured so that the customer has minimal downside. It's fully plug and play. And then the conservative move ultimately becomes, why wouldn't we bring this new technology in?

AI assessment note: “it's partially because the downside is asymmetric”

Q is that critical materials is one of actually the bright spots, particularly in sort of the climate energy conversation, that there's still a focus and still a recognition by this administration that that's something that we need to work on. So is that, is that your feeling? Like, do you, do you, are you feeling the love from the federal, like, Funding and sort of regulatory context, or not necessarily?

A Not yet. We have to find a giant pile of government money that magically helps us. But that said, I think there's a lot of real momentum here, and it's great. The structural logic at the end of the day is going to hold regardless of who's in office, because you can't electrify anything. You can't build AI data centers without copper, and much of the world's current refining and processing capacity sits outside the US. So The recovery argument is pretty simple. You can add domestic supply without new permitting, which makes it the fastest lever that the country has. The incentives, as they continue to emerge and evolve, I really look forward to that, but at the end of the day, the unit economics have to clear, and it has to make financial sense for mining customers.

AI assessment note: “Not yet. We have to find a giant pile of government money that magically helps us.”

Q so we go back to materials. Your whole thing is like the winner is going to be able to produce more materials than their adversary. I still don't think I completely understand the way you think about achieving that goal. Is the hardware that you're manufacturing has to be as simple as possible and then you're offsetting all the innovation in software? Is that, is that still the basic thesis?

A The idea is that you build hardware that is good enough, um, and build it, um, in such high numbers that you eventually have economies of scale, but also the ability to overcome any Um, you know, adversary just through pure mass. And the way you achieve that is that you, you need to be able to mass produce, you need to design to mass produce, but then crucially you need software to, um, to allow those systems to first of all, be smart and, and be able to To coordinate, um, but you ideally also absorb a lot of the complexity of the, of the hardware that usually, you know, would be these exquisite systems that legacy primes will make into your software so you can make the hardware simpler. That means that your sensors are simpler and you regain a lot of the accuracy through very good AI algorithms. It might also be that your, um, your manufacturing, um, tolerances are a little bit bigger and you regain a lot of the control and accuracy again Through software that might recalibrate the system as it goes. And, and this is what we do. And that works extremely well, but you have to think it through as, as one unified system. And once you do that, you can go into mass manufacturing. So the, the HX-II is now a mass manufactured strike drone. And like we had to go through like all of those learnings in, in July and August last year, where, um, we had to go from dozens of HX-II per month…

AI assessment note: “absorb a lot of the complexity of the, of the hardware... into your software”

Q mentioned, um, like, like, uh, roto-molding or rotational molding, because, and I think most people can, can picture what an igloo ice chest is like, but what made this different was the plastic was heavy duty. It's like, almost like industrial strength plastic that was So strong, you could stand on it, and you could sit on it, and it just, and it was heavier. Is that a fair description?

A Yeah, it, it was heavier. It's, rotomolding is a, is a manufacturing process where you take, um, polyethylene, which is a plastic pellet form resin, and you put it into a hollow mold, and you begin to coat the walls, and it really piles up in the corners and makes for a real durable part, and the overall wall thickness is quite a bit larger as well. You know, I remember in the early days, and when a customer called up and said, well, tell me about your product, what makes it durable, I'd say, It's roto molded. And I said, it's the same process used to make whitewater kayaks. And all of a sudden they get that imagery of a, you know, heavy duty, single piece plastic part.

AI assessment note: “Yeah, it, it was heavier. It's, rotomolding is a, is a manufacturing process”

Q this is still pretty early days on YouTube when people still thought of it as cats, you know, cats on skateboards, but you put out a YouTube. Do you, do you remember what the thinking was behind that? Because again, you know, you're still, you're growing, but you don't have a massive budget for advertising. And what did you think you were going to do with YouTube at that time?

A Well, I, you know, I felt like we could create some really short videos talking about the value proposition of, of Yeti. And I had a high school buddy that owned a camera and some audio equipment. And, uh, then we started getting into some, you know, funner videos of my friend, the same guy that owned the camera, found this bouncer down on, on sixth street here in Austin and hired him to come do this video where It's talking about the difference between an igloo cooler and a Yeti. Yeah. And it's this, you know, 500 pound man versus, you know, a Yeti. It shows him tearing up an igloo, right? Slamming it on the ground, stepping on it, hinges breaking, lid caving in, and then he tries to do the same thing to a Yeti tundra, and he's throwing it off cliffs, he's jumping up and down on it, and just, there's no better way to share a durability story than that kind of funny content.

AI assessment note: “I felt like we could create some really short videos talking about the value proposition”

Q What is, in, in the modern day, but prior to, you know, revolutionization by AI, what is utility resource planning actually look like inside the utility?

A Yeah, there, there's many parts of planning. When you say resource, it might be integrated resource on the bulk power system, or with, um, centralized generation, there's transmission planning , there's distribution planning, then there is DER planning. Each one of these are on silos, um, and recently just talking to a very large East Coast IOU, even transmission planning, there are like 12 departments doing their own thing. And each time a study is done, it's completely separate and siloed, so each use case is like a study. You want to connect a generator, it's a study. If you want to connect a load, it's a different type of study, but the underlying model is still the same. So studies today, I would say half if not more of the time, especially for distribution, is on cleaning up data. So data is in a ton of different places. Data quality is not too par, and a lot of manual effort is required to pull data together. Then you run the analysis, which is really tuned for a worst-case scenario planning. What's the five hours of the year that's going to be worst case for the next 10 years, and you plan to that level of standard. Now, people are improving, like looking at 96 hours per year, two 88 hours per year, five 76 hours per year, so each of those are like high, best case and low cases on weeks of the season or month over the years, and I would say the gold standard today is pr…

AI assessment note: “Each one of these are on silos... half if not more of the time is on cleaning up data.”

Q on their own. Uh, and the frontier is, is certainly not that far in front of open weights. Um, and so I, I'm curious to hear your perspective about what we should be thinking about in terms of, you know, what's gonna happen when companies that are not as scrupulous, you know, have access to this same powerful technology, and do we get into trouble in that, in that area?

A Um, yeah, so we can sort of see this coming and relatively soon. I don't know what the gap is. You would say, you know, six months, 12 months, maybe, uh, at the most between the closed wait frontier and available open source model. So it seems to be that, um, The open source models will very soon, if not already become capable of lending meaningful assistance, um, to destructive uses that, um, some people might pursue, uh, already cyber offensive capabilities has been a concern, right? With mythos, for example, that was withheld for that reason, but also, um, Say in biological weapons design, or chemical weapons, or other malicious uses. Um, and so it, It seems that you either need to prevent open weight models from being developed and released, or which might be better and more realistic, try to shore up some of the alternative defenses. For example, with bio, you could imagine regulating some of the other necessary inputs, um, DNA synthesis machines, for instance. So maybe it will be the case that there will just be widespread access to models that can help you design new pathogens. Um, and then you need something else to prevent that from actually resulting in a release of biological weapons, and that seems like DNA synthesis machines would be one excellent place to maybe, you don't need every lab to have their own DNA synthesis machine. They could have DNA synthesis as a se…

AI assessment note: “open source models will very soon, if not already become capable of lending meaningful assistance”

Q Well, and beyond that, I mean, let's go back to Hindu Kush, which, I mean, I think most people know you for MTV, such an iconic stretch of your career. Um, but you, you started by building an apparel brand out of India, right? And Afghanistan. I mean, Um, where it's not even clear what the rules were at times, right?

A I wanted to live there. I, I had been traveling, and I had developed this fascination with that part of the world. I figured, how can I afford to live here? I need to start a business, because I couldn't get a job, so I started this business in the garment business, knowing nothing, never wanting to be in the garment business, but it took off like a rocket, and then at the end, there was an embargo That Jimmy Carter put down that ultimately put my company out of business. Yeah. No more imports from India. And I sent three tons of clothes to, to, to Montreal and we smuggled them over the St. Lawrence Seaway, which was insane. I don't know what I was thinking, but, uh, I was kind of just looking for a little justice.

AI assessment note: “I wanted to live there... I need to start a business... in the garment business”

Q If I'm the founder of a company, an early-stage company, do I just accept that I'm gonna have B-tier or C-tier AI talent? And I don't mean that denigratively or rudely or horribly, but they're an anthropic and open AI. I mean, Google can't freaking keep.

A I think it's the wrong, um, framing, because if you think about it, I mean, look, when I was, you know, when you were building a software company in the age of the PC, You had four-tier chip talent because you weren't building a chip, right? The point is, if you're an AI company and you feel the need to build a frontier model, then yes, you've put yourself in direct competition with someone, and if you don't have the good people, you're toast. So, what you got to do is make the model a compliment and have A-tier talent at UI, A-tier talent at, you know, AI implementation, A-tier talent at the things that you have your competitive advantage in.

AI assessment note: “I think it's the wrong framing... make the model a compliment”

Q I'd like to make some extra money. What's an idea that most people have not heard about?

A For homeowners looking to make extra money and, wait for it, tax-free in many cases, I have an idea teed up for you. You can rent your own home tax-free if you follow these guidelines. The rule is referred to as the fourteen-day rule. It's sectioned, two-eighty-A of the federal tax code. You can use your home with all the tax breaks associated with home ownership And you can rent it out free of federal taxes if you rent it for fewer than 15 days in a calendar year. The rental income is generally excluded from your federal income tax, but check with your tax advisor. I think you may not even have to report it. People call it the Augusta rule because homeowners in Augusta, Georgia for many years have rented their homes To masters golf fans and kept every single penny, but the rule works anywhere in America and works especially well where a town throws a big event, but doesn't have the hotel rooms for everyone. So for example, Coachella, the Kentucky Derby, the Superbowl, the F one race, which is coming up in Las Vegas. If you own a nice home near that track, you can clear out for race week. Rent something inexpensive in the desert nearby, and lease your house for some serious money, and generally owe the IRS nothing on it. Same house, same week, the difference is knowing the rule. Now let's get into the fine print, because this rule has a sharp cliff, and the cliff, the hard limi…

AI assessment note: “You can rent your own home tax-free if you follow these guidelines.”

Q Should I hold my rental and an LLC?

A I do get this one constantly, and people expect the attorney to say always, and the honest answer is, it depends. And here's my idea of the framework. What an LLC actually does, it separates the rental's liabilities from your personal assets. That's important, but it works only when it's set up right and run right. And that includes a separate bank account, Real records, and no commingling. If you treat it like your own bank account, you lose the separation. So for example, if the rent checks go into your own personal checking account, not the LLCs, if the repairs go on your personal credit card, these are the issues that if somebody sues, the lawyer holds them up and says, look, this person, this isn't a real company. This is what they call an alter ego. That's a legal term for treating it like yourself.

AI assessment note: “the honest answer is, it depends. And here's my idea of the framework.”

Q Yeah, makes sense. And, and meanwhile, you guys are all obviously getting older. So tell me a little bit about your involvement as kids with, with his business. What would you guys do? Salma, what do you remember? Would you guys go there on the weekends and like pack boxes and help out?

A Yeah. I remember my first job in the summer was actually going into the factory and stickering bars because I didn't have packaging machines at that time to do it. So I remember me and my sister would be lining them up and we'd race as who can do the more, you know, who can sticker the most amount of, um, fruit and nut bars. So that was our, that was my first exposure into the, into the family business and on the factory floor. And we'd work right. Eight in the morning till four in the afternoon with one half an hour of lunch, but we'd be on our feet all day, and we'd get five dollars cash at the end of the day.

AI assessment note: “my first job in the summer was actually going into the factory and stickering bars”

Q What can you actually prefab? Like what I'm trying to, what are the Lego building blocks?

A Yeah. Yeah. Good question. So, um, almost everything in the electrical world and almost everything in the mechanical world are prefab for us now. So think of all the way out from where the utility comes onto the campus and I've got a transfer switch or I've got a transformer that I'm stepping down. We'll have all of that pre-assembled on a, on a skid so that you don't have to have somebody wired between the transfer switch and, and Wire between that and the, you know, a transformer. Um, we'll have the entire power center, you know, everything in there that distributes electricity into the building all show up in one unit. Um, we have our entire, uh, today, both liquid or hybrid cooling, so both liquid and air, shows up in a single packaged unit. So these things, you're not having to put together any of it on site. Um, all of our buildings are tilt up You know, um, prefabricated so that the panels show up and we literally just crane them and set them in place so no one's having to make any of the walls, no one's having to pour any, uh, forms or, or build any forms or pour any concrete. They all show up on a truck, again, literally just like Legos. Slab A goes in slot A. Slab B goes in slot B. And, uh, they get lifted off a truck and slid into place. So almost the, uh, the roof is the same way. Uh, we have prefabricated double T's that support the span. Literally the whole buildi…

AI assessment note: “almost everything in the electrical world and almost everything in the mechanical world are prefab”

Q we had a nice launch of the book. Very successful. But before we get into all that, I want to start off with a fun question for you. Ok, you're expert inference engineer. What happens when I send a long query, say, 200,000 tokens into base tens inference? What's the process of query through GPU model routing, balancing, all that? What is all the stuff that we don't think about?

A With a long query specifically, the first thing that I'm going to ask is, have you sent me this query before, or at least part of it? Um, and I really hope you have, because it's going to be a lot easier for me and a lot cheaper for you. So the first thing that we're going to look at is some kind of cache away routing where we're going to see, we probably have a number of instances, a number of replicas up serving whatever model you're hitting. Uh, we want to send this one to Something with number one, available pre-fill workers, and number two, ideally, some cached input already there so that we can skip pre-fill on at least part of these 200,000 tokens. Um, if you're doing 200,000 tokens, it's probably coding or a multi-tone agent or something where you would expect to have that cached. Um, if you don't, we're gonna have to send it to a pre-fill worker. Um, we've, at least on certain models, disaggregated pre-fill and decode. Um, so You're going to have one set of GPUs that's solely going to process the input, create that KV cache, and get you your first token, and then that's going to be passed over to a separate set of GPUs, which is going to run decode. We're going to iteratively make those tokens. Um, we're probably going to have some kind of speculator model in front of that. I'm going to assume that you're doing coding, and because of that, our speculator model, which a…

AI assessment note: “the first thing that we're going to look at is some kind of cache away routing”

Q know, it's the same thing, it's just a model. Can we not just distill a lot of the weights, quantize the speculator, but out of my domain. Um, I guess the, the question that also comes up is, this is all for big server workloads, right? How much of this applies to, say I have this MacBook, I want to run Gemma really efficiently, um, Similar problems? Not the same?

A Pretty different. I talked to Cero, um, about this on, on his podcast a couple weeks ago. The difference between inference engineering for the data center and for production workloads versus inference engineering for local AI is that we start with fundamentally like different constraints and different goals. With local AI, it's how do I fit this model onto my hardware? And then make it less dumb. And with data center inference, it's how do I load this model and then make it less slow? And obviously, you know, we care about less dumb and they care about less slow, but the local AI inference engineering ecosystem, I think actually has a lot for us to learn from in the data center space. They are experts in various forms of quantization, including dynamic quantization that we just kind of don't touch. In the pruning, in the distillation, in the, uh, you know, layer removal. There's removal letters less. Yeah. Yeah, but, but,

AI assessment note: “Pretty different. The difference between inference engineering for the data center and local AI”

Q Well, that was incredibly succinct. Thank you. Normally people take about four hours after I ask for a succinct description. When you look at the models that you have on ARENA, the sheer number of them, Bluntly, I just am faced with the one question. Holy shit. Is this like the true commoditization of models? Are they just a complete utility layer at this point?

A Well, I think that there's, ah, the, the big question around this has started to rise because of open source models. So I think if you were to only look at the closed source models, you would say there's acceleration, but it hasn't quite commoditized yet because that layer is still owned by a pretty small group of companies. It would be an oligopoly if we only had the closed source models. But what seems to be happening is that the open source models, especially from China, Have really rapidly improved. And for the first time ever, we saw a couple of weeks ago that Kimi K three actually beat the best closed source American models, uh, on a, you know, pretty important subset of tasks, for example, front end, front end coding, like web development, which a huge fraction of developers are web developers.

AI assessment note: “if you were to only look at the closed source models... hasn't quite commoditized yet”

Q Were they selling them themselves, or there's, like, vendors inside Walmart?

A They had vendors, and they had partners, and they had their own Walmart card. It was a whatever card. So I wrote them in January before COVID and said, listen, if you're in Bentonville, I'd love to go see you. And lo and behold, he said, yes, shut up, see them with, with Doug. And he introduced me to John Furner, who's now the CEO of Walmart, who's alongside with Doug. They were there for almost 30 years at Walmart, both of them, since they, since they were 17. And John is my age, so we're, we started at the same time in two different continents, and we were still in the same room talking about this, and I said, listen, you guys can build something much better with financial services here. This is not your priority number one, two, or three. It is for us. Let's figure it out. And they did something that never, they had never done. They created a JV, a company that we both go on, and we went together and built a rockstar team We have a new brand. We bought two companies and to build the infrastructure run by Walmart Smile today, and it's called OnePay, and it's one of the biggest financial brands that you've never heard of.

AI assessment note: “They had vendors, and they had partners, and they had their own Walmart card.”

Q at, you amortize it across a much, much smaller project, and that blows out the economics and it kills you. Um, this is where I think, you know, the way that you've been thinking about it is the most interesting to me, because that's the, this is the problem that's been the most intractable in my mind. So what, what gives you confidence the soft costs are going to change?

A Yeah. All right. So, If you think about sort of the total project cost of a C&I battery, right, roughly half of it is what we traditionally call soft cost, right? So let's just say for the sake of argument, uh, 800 dollars a kilowatt hour is sort of the average C&I project in the world today. So traditionally, the way to think about it, and this is sort of post-ITC, right, is three to 400 dollars of that is the hardware. About a hundred dollars a kilowatt hour of that is the soft software. So 400 dollars a kilowatt hour gets you the hardware and the software obligations you need, right? Um, the other 400 dollars a kilowatt hour is what we traditionally call soft costs, and that's broken down into two buckets. About 200 dollars a kilowatt hour of that is installation costs, and about 200 dollars a kilowatt hour of that is what I generally call transaction costs, right? Which is all the things you mentioned. Interconnection, permitting, financing, You know, the people, cost of customer acquisition, all those types of things. Um, Like, I think there are huge opportunities for reduction, both on the installation side and on the transaction cost side. The thing I've really been focused on over the last, you know, 18 months is the transaction cost side of things, and that's where I think AI is a huge part of the solution, right? And so, you know, the reality is, is that a lot of the …

AI assessment note: “train agents to do those repetitive workflows, and that drops your transaction costs significantly”

Q In addition to some of the things we talked about, industrial assets, a little bit of infrastructure, defense, what are some of your favorite themes in the portfolio today?

A Another sector we love is senior housing. We've been very active in that. It goes all back to structural drivers. We know the eighty-plus age cohorts growing at nearly five percent a year while the overall population is dead flat. It's not only where all the growth is, it's where all the wealth is, too. There are a lot of them with a lot of money. That translates to a lot of demand and need for senior housing. That's a space we've been focused on because of the demand side, coupled with a dramatic drop-off in supply. Another one is net lease. Net lease is a lease structure in which the tenant pays the rent and all the expenses. It's the type of real estate where you, as the asset owner, have the most predictability of cash flow, the most downside protection between long-term contractual cash flow tied to credit tenants, tantamount to a credit investment, but you also have the hard asset ownership. The benefits of that real estate ownership that give you the inflation hedging, the appreciation potential, the ability to play these long-term mega trends for individual investors, tax efficiency. That's been a very big focus for us as well.

AI assessment note: “Another sector we love is senior housing. We've been very active in that.”

Q You mentioned that when you first started investing, there was already a platform here, Morgan Stanley, we'll say, 26 years ago. How is that real estate part of what you've been involved with for so long, evolved over the years?

A When I started, Morgan Stanley was one of the first amongst the largest real estate private equity businesses. We've seen the space institutionalized pretty dramatically. If I think back to 2000, the real estate private equity business was largely opportunistic closed on funds and U.S. pension fund capital. We've seen over time a diversification of the types of investing. More core, open-ended, or different strategies. We've seen a diversification of the investor base. It's super global today. Our business was a leader in a few regards there. We were one of the first to say, we want to serve our investors more holistically. We were one of the first entrants from the opportunistic space into the core space. Fundamentally, we're in the client service business. We first and foremost focus on what role should real estate play in investors' portfolios? What is most important to them? Ultimately, having strategies that serve their purposes, having core and opportunistic, having a few core plus strategies, being global in what we do has given us much better perspective. It's enabled us to have better judgment across the strategies and attract extraordinary talent as well.

AI assessment note: “We've seen over time a diversification of the types of investing.”

Q And tell me how it works. Like I'm a senior citizen. How does it work?

A Super simple. So guy, you would pick that after breakfast, you want to check in with snug every day. Let's say at 8:30 in the morning, we'll send you a bunch of reminders. And if you don't open your snug app and hit just the basic check mark by eight 30, we will then send you a few more reminders. Then we will notify your emergency contacts. Hey, Guy did not check in this morning. Here is his last known location. Um, and that's on our free plan. If you are on our paid plan, we have a alarm service that will call you as well, call your emergency contacts, and then if still no one confirms you're okay, arrange a visit by emergency services.

AI assessment note: “you would pick that after breakfast, you want to check in with snug every day.”