Mar 26, 2026 · 1h 14m · in-depth
How to build a company when you’re not an optimist | Jay Kreps (Co-founder and CEO, Confluent)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this in-depth interview, Confluent co-founder and CEO Jay Kreps details the journey of creating Apache Kafka, commercializing open-source infrastructure, and navigating the strategic pivots required to scale a public cloud enterprise. He shares tactical insights on founder-led sales, technical category creation, and leading through uncertainty with disciplined tenacity rather than blind optimism.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Brett holds 12.2% of the talking time here. How this is scored →
Brett as informed peer 3.9 Guest teaching 5.9 Guest disagreement 1.3 Brett pushing back 1.4
speaking balance: gold is Brett, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 49:25 Pushing back against internal and investor consensus on cloud
Jay forcefully dismisses conventional enterprise software thinking, recounting how half the company and their own investors told them building the early cloud product was a terrible idea.
Hardest push from Brett ▶ 4:11 Pushing for precision on the eighty percent executive knowledge ruleBrett immediately challenges Jay's arbitrary eighty percent threshold, demanding to know why it is not ninety, fifty, or twenty percent.
Biggest teaching moment ▶ 24:35 Masterclass on positioning infrastructure for public marketsJay delivers an insightful explanation of product marketing architecture, explaining how anchoring on databases to establish 'data in motion' unlocked public investor comprehension during the IPO.
Brett holds their own ▶ 43:40 Synthesizing software lock-in and pricing power dynamicsBrett demonstrates strong domain expertise regarding software switching costs and pricing leverage, prompting Jay to formalize the economic game theory between vendors and enterprise buyers.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Brett as informed peer | Guest teaching | Guest disagreement | Brett pushing back | Why |
|---|---|---|---|---|---|---|
| Transitioning From Software Engineer to Public Company CEO | 4 | 5 | 1 | 2 | Brett asks targeted framing questions about transitioning from engineer to CEO and probes Jay's specific '80%' metric. Jay explains the shift from deterministic engineering logic to operating in a fog of partial understanding. | |
| Designing Context-Specific Go-to-Market Strategies and Sales Motions | 4 | 6 | 2 | 2 | Jay breaks down why copy-pasting go-to-market playbooks fails across different genres of software companies. Brett probes whether early go-to-market requires pioneering talent or prolonged founder immersion. | |
| Origins of Apache Kafka and Distributed Systems at LinkedIn | 3 | 6 | 1 | 1 | Brett invites the origin story of Kafka at LinkedIn. Jay educates on the historical landscape of distributed systems, machine learning constraints, and the bottleneck of moving data at scale. | |
| Internal Deployment, Team Formation, and Open-Sourcing Kafka | 3 | 5 | 1 | 1 | Jay recounts the internal organizational politics of replacing legacy pipelines at LinkedIn and the disappointing lack of initial external traction upon open-sourcing Kafka. | |
| Unlocking Developer Adoption Through the Seminal Kafka Blog Post | 3 | 6 | 1 | 1 | Brett highlights that adoption shifted before product code significantly changed. Jay explains how his 25-page seminal blog post served as critical product marketing that unlocked developer understanding. | |
| Structuring Product Marketing and IPO Messaging for Investors | 4 | 7 | 1 | 2 | Jay details the conceptual pyramid of product marketing and how positioning Confluent around 'data in motion' vs. 'data at rest' enabled public investors to grasp their total addressable market. | |
| Evaluating Commercialization Paths and Spinning Out Confluent | 4 | 6 | 2 | 1 | Jay reflects on founding Confluent in 2014 when open-source monetization had a poor track record and AWS dominated early cloud, criticizing top-down market analysis over problem immersion. | |
| Navigating Accidental Seed Fundraising and Series A Financing | 3 | 5 | 1 | 2 | Brett asks about the sequence of fundraising and product execution. Jay describes how LinkedIn's offer to invest forced an accidental, un-incorporated venture raise that evolved into a Series A. | |
| Bootstrapping Enterprise Software Sales and Hiring Early Reps | 4 | 6 | 1 | 1 | Jay describes early enterprise selling, closing six-figure deals without full product scaffolding, and hiring individual contributor sales reps willing to operate without playbook structure. | |
| Commercial Open Source Dynamics, Pricing Power, and Retention | 5 | 6 | 1 | 1 | Brett proposes that data infrastructure has natural lock-in. Jay reframes this through game theory, explaining the balance between open-source commodity pricing and proprietary differentiation. | |
| The High-Stakes Strategic Pivot to Confluent Cloud | 5 | 7 | 2 | 1 | Jay recounts the painful, existential pivot to Confluent Cloud despite internal resistance, investor doubts, and churn, explaining the critical distinction between what a company 'can do' versus what it 'must do'. | |
| Founder Tenacity, Pessimism, and Pursuing Strategic Imperatives | 4 | 6 | 2 | 2 | Jay discusses non-optimist founder tenacity, describing how accepting low probability odds and enduring prolonged pain outlasts competition across strategic multi-year imperatives. | |
| Cross-Functional Problem Diagnostics and Challenging Planning Assumptions | 4 | 6 | 1 | 2 | Jay explains how CEOs diagnose cross-functional breakdowns and push back on two-year team estimates by forcing teams to imagine rapid three-month iterations. | |
| Scaling Culture, Decentralized Autonomy, and Preserving Excellence | 4 | 6 | 1 | 1 | Jay shares broad company-building wisdom on scaling, contrasting fine dining autonomy with corporate 'Chipotle' mediocrity, and empowering decentralized product units with explicit P&L accountability. |
Statements from this episode (22)
Kreps: Engineering decisions are knowable, early startup decisions are unknowable
“Certainly for engineering decisions. They're mostly knowable and there's mostly kind of right and wrong answers, but usually, especially the early phase of the company, you're making a lot of kind of very big critical decisions with a lot of unknowable aspects…”
Kreps: CEOs must learn ~80% of what functional executives know
“I think CEOs need to know, you know, about 80% of what their executives know about their function, I think, over time. So really kind of learn that discipline. Not enough to be good Add it, but enough to be, to know what good is and know if it's going well.”
Kreps: GTM construction is where early startups fail the most
“I think probably for early companies that go to market construction is, is probably one of the areas where this goes the most wrong. Where you're effectively, you're trying, you're talking to people and you're trying to cut and paste what they've done into you…”
Kreps: Half the challenge in ML products is data infrastructure, not models
“Half the problem in any of these products is actually not the machine learning stuff. It's actually just the data, like Getting, getting it, being able to work with it at scale, being able to apply it, being able to do that in the context of like a real runnin…”
Kreps: Signal of a major problem is use cases growing tenfold
“You can kind of, you know, you're onto a big problem when the number of use cases just keeps, you know, going up by like an order of magnitude that kind of pop out and where the problem you're solving seems really fundamental.”
Kreps: Apache Kafka Sat on the Shelf During First Open-Source Year
“And you know, it actually took a while, you know, probably the first year, it mostly sat on the shelf as a open source project, which we were quite disappointed about.”
Kreps: Open-Source Projects Require Product Marketing to Drive Adoption
“What was it that it took us probably several years to discover was basically product marketing. You know, in an open source context was, yeah, you gotta tell people why it's interesting.”
Kreps: Confluent coined 'data in motion' against databases for its IPO
“Our approach to that was for the IPO was very much like, okay, databases, data at rest. But now all the parts of the company are connected. There's going to be an equally important problem of data in motion. And so, you know, the goal was to convey, Hey, these…”
Kreps: Growing companies inherently lose their communication edge by addressing multiple audiences
“Usually as companies, as time goes by, as a result of that, many things become more watered down because you're trying to say something. To a bunch of people who have very different contexts. And you notice this, that companies kind of lose the edge in communi…”
Kreps: LinkedIn offered an unprecedented balance sheet investment in Confluent
“When we went to quit, LinkedIn was like, oh, okay, well, this is, you know, first of all, we don't really want you to leave, but if you're going to leave, we do think this is really valuable. We would be interested in investing.
And we were like, okay.
Off the…”
Kreps: Confluent's initial seed round shifted to a Benchmark-led Series A
“And it went from something where LinkedIn was going to be the kind of major investor to being, you know, a smaller part of the round. And it went from kind of a seed investment to a series A through the course of that fundraising process. But yeah, it was, you…”
Kreps: Confluent's one-year pitch roadmap took five years to complete
“What we had pitched was actually very close to what we built. It's just, I said we were going to do it in a year. And I think in practice, The first thing we launched was much less than that. And to do everything on the slide took like five years.”
Kreps: Amazon built Kinesis to imitate Kafka before monetizing the open-source
“Amazon had a system that was actually built in, I believe imitation of Kafka called Kinesis. So it was incompatible, but kind of looked roughly like it. And that was popular. And we were like, oh man, that's going to take the opportunity. And if it doesn't, Th…”
Kreps: Startups can amplify founder-led sales by hiring early-stage IC reps
“And I think that's really a worthwhile investment of, you know, I think you can kind of amplify The founder led sale thing quite significantly with just a few people. But you need people who have been successful in early companies where they just.”
Kreps: There is no business in selling pure open-source software
“And so, you know, if you're trying to sell a product which is purely open source, you know, effectively The price of that product should collapse to the cost of providing it, which is basically kind of a scale of zero, right? So, so there's no business in sell…”
Kreps: Extreme vendor lock-in lets infrastructure vendors over-extract value
“I think it is the case that some of these early infrastructure offerings, you know, that it goes too far where the lock-in is so tight that the vendor can actually, you know, extract in some sense more than the value they create because moving off of it is suc…”
Kreps: Existing Customer Pressure Naturally Starves Second Products
“I think inherently that a second product is always. Kind of irrelevant to people's main goals, you know, and the, all the pressure of a business is to satisfy the customers that you have, the big prospects coming in the door, that, that kind of main flow of bu…”
Kreps: Confluent relegated its core cash-cow product to engineers' spare time
“Originally it was like a small group that was working on the cloud offering. And then we were like, no, okay, everybody. Only does the cloud thing, and we will work on the software product in our spare time. Team by team, which is a very dangerous thing to do,…”
Kreps: Confluent struggled for years to expand into data processing
“One of the more recent versions of this was we felt is very important for the company to do the processing of data, not just the flow. And we worked on this largely unsuccessfully for a number of years.”
Kreps: High-level company problems are inherently cross-functional
“Anything that's not working
At a high level is inherently cross-functional. And you know, you'll see a weird phenomenon where people in each function are often don't have enough global context to totally diagnose it.”
Kreps: Demanding a three-month timeline forces engineering teams to break assumptions
“If you kind of just go back and say, well, okay, you know, what would the three month version of it look like? Maybe that's totally impossible, but a significant portion of the time, there is something you could do in three months. And what would that be? Is t…”
Kreps: Managers who recruit their own teams are massively more successful
“So I found that to be very true in hiring where, you know, people who just, you know, act like they have, they, you know, they just have to fight for their life to go hire this team by themselves as if there was no recruiting support and they're just really de…”