Sep 27, 2024 · 1h 1m · mad
AI at Datadog: Monitoring machines in the age of LLMs | Olivier Pomel, CEO of Datadog
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of The MAD Podcast, host Matt Turck interviews Datadog co-founder and CEO Olivier Pomel on Datadog's growth into a multi-billion-dollar cloud observability and security platform. Pomel shares insights on multi-product development discipline, real-time data engineering, and Datadog's pragmatic approach to AI and time-series foundation models.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 24.6% of the talking time here. How this is scored →
Matt as informed peer 3.7 Guest teaching 3.9 Guest disagreement 0.7 Matt pushing back 0.9
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 59:55 Critique of Founder Mode viral trend
Olivier explicitly pushes back against the viral trend of Paul Graham's Founder Mode essay, cautioning that taking short essays out of context leads to misuse across tech ecosystems.
Hardest push from Matt ▶ 35:17 Challenging cloud dominance with on-prem AI trendMatt directly challenges the premise that all AI workloads will run in public clouds by citing the movement to bring AI models to on-premise data centers and dedicated hardware.
Biggest teaching moment ▶ 8:28 YC rejection feedback on platform vs productOlivier educates Matt on positioning dynamics by recounting Paul Graham's explicit YC rejection note explaining why abstract platforms fail without a concrete, focused initial product.
Matt holds his own ▶ 40:53 Recalling guest's historical AI skepticismMatt demonstrates deep expertise and preparation by calling out Olivier's specific past statements regarding statistical additions/subtractions disguised as AI and false positive concerns.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Origins in New York and the Evolution of the French Tech Ecosystem | 2 | 2 | 0 | 0 | Matt engages in friendly introductory banter about French technology history and relocating to New York. Olivier explains his background at IBM and why he stayed in NYC during the dot-com crash. | |
| Unifying Silos and Navigating the Tool-vs-Platform Journey | 3 | 4 | 1 | 1 | Matt queries the venture capital cliché of building a tool before a platform. Olivier recounts getting rejected by YC and Paul Graham's message about needing a initial core product before claiming platform status. | |
| Product Expansion: From Metrics to APM, Logs, Synthetics, and Security Consolidation | 4 | 4 | 1 | 2 | Matt probes whether security expansion was customer-driven or a strategic shift. Olivier explains that security is fundamentally a developer and operations problem rather than just threat-hunting. | |
| The Methodology of Multi-Product Innovation, Pricing Gates, and Culture | 3 | 5 | 1 | 1 | Matt asks about internal processes for validating new product offerings. Olivier contrasts Datadog's customer design partner iteration against Apple's secretive R&D model and details pricing gate methodology. | |
| Measuring Product Success, Short Feedback Loops, and Developer Productivity | 4 | 4 | 1 | 1 | Matt links software evolution to DevSecOps trends. Olivier details why short month-to-month contracts provide vital feedback loops compared to multi-year enterprise deals. | |
| AI Tailwinds, Cloud Acceleration, and LLM Observability | 5 | 4 | 1 | 2 | Matt questions if AI compute on-premise threatens public cloud trends. Olivier reframes on-premise AI infrastructure as behaving essentially like cloud environments. | |
| Datadog's Approach to Integrated AI and Automation | 5 | 3 | 1 | 1 | Matt references specific past conversations regarding Datadog's caution around AI hype and false alerts. Olivier confirms the philosophy of avoiding overpromising to maintain user trust. | |
| AI Models and Data Types in Systems Observability | 3 | 4 | 0 | 0 | Matt asks about model types for observability data. Olivier explains moving from non-transformer statistical models to incorporating multi-modal context such as Slack channels and documentation. | |
| Introducing Toto: Datadog's Foundation Model for Time Series | 3 | 4 | 0 | 0 | Matt asks why Datadog developed its own time series foundation model named Toto. Olivier outlines how rich operational metadata enabled Toto to achieve state-of-the-art benchmark results. | |
| Deep Dive into Watchdog for Automated Anomaly Detection | 4 | 4 | 1 | 1 | Matt explores chatbot fatigue and agentic workflows. Olivier discusses shifting from conversational UI bots to proactive autonomous incident response agents without turning into Clippy. | |
| Architecture of Datadog's Real-Time Data Platform | 3 | 4 | 0 | 1 | Matt asks about platform scale behind handling trillions of metrics. Olivier explains Datadog's engineering practice of constantly rebuilding underlying modules. | |
| Technical Deep Dive: Custom Event Stores and Husky | 4 | 4 | 0 | 0 | Matt checks his understanding of data flows and log management. Olivier explains decoupling storage from compute in custom event engines like Husky to manage cost growth. | |
| Structuring AI R&D: Avoiding Isolated Labs | 5 | 5 | 2 | 2 | Matt asks about isolated R&D labs and brings up Paul Graham's Founder Mode essay. Olivier warns against isolated labs and critiques how Founder Mode is easily misapplied and abused. |
Statements from this episode (28)
Pomel: European tech ecosystem remains fragile without large-scale companies
“I think we still have to see them scale to large companies in, in France, in Europe more broadly. It hasn't happened yet. So the ecosystem is still young, and I would say still fragile from that perspective.”
Olivier Pomel: Datadog's largest enterprise customers pay tens of millions annually
“Our smallest customers, you know don't pay us anything, and they're individuals or students. And our largest customers are the largest companies in the world, you know, paying us tens of millions of dollars a year, and we'll have pretty much everything in betw…”
Olivier Pomel: Y Combinator rejected Datadog over lack of initial product
“We applied to all sorts of incubators, we got to a Y Combinator interview, and then we didn't get into Y Combinator. I have an email from Paul Graham that says You know, a platform is only as good as its first product, and you don't have a first product, blah,…”
Olivier Pomel: Startups must frame products within existing categories and spend
“You actually have to take your customers where they are and talk to them in terms that they understand in terms of their existing categories, their existing spend, existing solutions.”
Pomel: Within five years, security will inevitably attach to observability
“We have a thesis that security, like, five years from now will be will be, or rather, let me rephrase it will be a no-brainer that you have to attach your security to your observability, because that is what gets deployed everywhere in your application, in you…”
Pomel: Cybersecurity software will consolidate into large platforms
“We think that it's too complex. Like, it's impossible for the customers to actually understand how to piece that together, and to integrate everything into a consistent whole that really protects them, and so we think that security is going to go through the s…”
Datadog rejects Apple-style secret product development
“We don't do it like Apple, like we don't disappear in a basement for three years and then, you know, ship a fully formed product that takes the world by storm. Instead, from the earliest days, like we work with design partners, we work with customers, and we t…”
Half of Datadog design partners drop off after pricing is introduced
“When the product is ready, when it's great, about half of the design partners just disappear.”
Datadog keeps struggling product teams small rather than killing products
“We did not kill products, but we, the way we do it is we start small, and we've waited a long time to scale up.”
Pomel: Negative customer feedback is the only feedback worth listening to
“The people who are used to scaling the products that work ignore the negative part, whereas it's the only part that's worth listening to.”
Pomel: Month-to-month contracts beat multi-year deals for validating new products
“So for new products and new companies, I would argue, month to month is great. Because your customers can try at any time which means you'll get the hard reality to hit you in the face and you can't ignore it, you know, which is a problem when you have, so you…”
Pomel: AI will boost developer productivity by 10x to 100x
“And I think AI on top of that is going to give us maybe another order of magnitude or two in terms of, you know, what a human can produce functionally.”
Pomel: AI shifts software value from writing code to understanding behavior
“Every time you add productivity, you add complexity, meaning that you have less and less of an understanding of what it is you're doing... So a lot of the value shifts from just creating that thing to understanding how it's going to behave, how it's changing o…”
Pomel: AI model observability will take years to fully flesh out
“So I think it's going to take maybe a few years for that category of observing the models themselves to fully flesh out in terms of what the use cases are, the needs and where they go.”
Pomel: IT teams won't manage AI separately from existing cloud infrastructure
“You are not going to manage your AI separately from your databases separately, from your network separately, from your security separately. Everything makes more sense when it is managed together and when we can assemble the full picture for you.”
Pomel: Datadog historically avoided 'AI' terminology, viewing it as 'bullshit words'
“For the longest time in the history of the company, we've been careful about using the words AI. We thought they were, you know, bullshit words mostly.”
Pomel: AI solves only 2% to 5% of systems management cases
“Right now we'd be lucky to solve, you know, two percent, three percent, five percent of the cases with AI.”
Pomel: AI might resolve 20% of IT issues in a year
“Maybe in a year it's going to be 20%, maybe, but it's going to be gradual.”
Pomel: OpenAI's latest models remain early stage for reasoning capabilities
“There's more we can do maybe on the reasoning side, though I would say even with the latest releases from OpenAI, it's still fairly early in terms of the quality of the models and what can be done there.”
Pomel: Datadog's Toto model beats other models on observability and weather benchmarks
“This model from day one was state of the art, like, it's it beats all the other models, of course, on the, on observability data, we have special benchmarks from that, but also for other things like weather data which was very surprising to us.”
Pomel: Anomaly detection systems must prioritize avoiding false positives over false negatives
“The most important, when you do things like that, is to not generate false positives. So these systems tend to generate more false negatives than false positives. Basically, if the system is not sure it's probably not going to tell you anything, because if it …”
Pomel: Product AI chatbots require handholding as users struggle with prompts
“It works great for some use cases, but you need to handhold the users a lot more. Like, users don't necessarily know exactly what to ask for and how to ask for it, or what to go next, you know, once they've asked a question.”
Pomel: Many big tech AI assistants risk being too much like Clippy
“The risk right now is that a lot of the assistants we're seeing from the big companies are too close to Clippy.”
Pomel: Observability log volumes grow much faster than customer revenue
“The biggest challenge with observability is that any application can generate any arbitrary large amount of logs. And so the data volumes grow much faster then our customers revenue”
Pomel: Datadog splits engineering equally between core platform and specific products
“So yes, the breakdown is roughly half of our engineering team is on the platform, and half is on the is assigned to specific products.”
Pomel: Datadog avoids dedicated AI labs because they set wrong expectations
“We, so we don't have a lab. In general, we're a little bit careful with labs. I think it's sets the wrong expectation.”
Pomel: Paul Graham's 'Founder Mode' essay will do more harm than good
“My worry about the founder mode, and everything that gets pulled down to a very short piece like that, is that I think it's going to be used and abused in all sorts of different ways, because there's so much context that goes into every single word in there, l…”