Dec 5, 2013 · 50m · mad
Fireside chat with Dwight Merriman // Data Driven #8 // Sep 2012 (interviewed by Matt Turck)
Dwight Merriman · 35m spoken Matt Turck · 6m spoken AB Mendez · 1m spoken Michael Selick · 35s spoken Andrew Dinsmore · 11s spoken
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this fireside chat hosted by Matt Turck at the NYC Data Business Meetup, 10gen co-founder Dwight Merriman discusses MongoDB's NoSQL database architecture, commercial open-source monetization strategies, and the growth of New York City's enterprise tech ecosystem.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 13.7% of the talking time here. How this is scored →
Matt as informed peer 2.2 Guest teaching 4.8 Guest disagreement 0.6 Matt pushing back 0.6
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 33:08 Dwight counters NY seed funding narrative
Dwight directly pushes back against Matt's suggestion that raising seed funding for NY enterprise startups is a major barrier, stating that perception has already changed.
Hardest push from Matt ▶ 8:33 Matt questions NoSQL replacement scopeMatt directly challenges NoSQL positioning by asking if it claims to be a total replacement for relational databases or if relational still makes sense.
Biggest teaching moment ▶ 3:56 Masterclass on hardware architecture and horizontal scalingDwight provides an in-depth explanation of modern processor architecture limitations and why commodity multi-server clusters replaced traditional vertical mainframe scaling.
Matt holds his own ▶ 45:06 Matt intervenes to contextualize Google SpannerMatt actively manages the interview by interrupting an audience question to ensure Google Spanner's paper premise is explained to everyone before Dwight answers.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Understanding NoSQL and Horizontal Scaling | 2 | 6 | 1 | 0 | Matt prompts Dwight with foundational questions on NoSQL and horizontal scaling. Dwight delivers an extended technical breakdown of hardware evolution, commodity cluster mechanics, and document-oriented data models. | |
| Comparing NoSQL Categories and Relational Use Cases | 3 | 6 | 1 | 1 | Matt probes whether NoSQL aims to completely replace relational databases and asks for categorization of NoSQL types. Dwight clarifies that one size no longer fits all and categorizes graph, key-value, and document stores. | |
| MongoDB 2.2 Release and Future Product Roadmap | 2 | 5 | 0 | 0 | Matt asks about the MongoDB 2.2 release and product roadmap. Dwight explains new technical features including the aggregation framework, concurrency lock improvements, and operational tool requirements. | |
| Open-Source Strategy and Business Model | 3 | 4 | 1 | 1 | Matt asks about open-source business strategies and community building. Dwight lightly reframes the timeline by noting MongoDB was open-source from inception, detailing market size and revenue compression dynamics. | |
| Transitioning from Startups to Enterprise Customers | 4 | 4 | 0 | 1 | Matt asks how MongoDB navigated transitioning from early web startup adoption to selling to enterprise clients. Dwight explains early adopter psychology and connecting enterprise needs to his past DoubleClick experience. | |
| Startup Opportunities in Big Data and B2B | 3 | 4 | 0 | 0 | Matt asks where untapped opportunities exist in B2B and big data. Dwight highlights Sequoia commentary on B2B underweighting and cites surrounding ecosystem opportunities like BSON and analytics tools. | |
| Enterprise Tech Ecosystem and Hiring in New York | 4 | 3 | 1 | 2 | Matt questions whether enterprise tech startups face fundraising barriers at the seed level in New York. Dwight reframes the premise, arguing that New York VC interest has caught up and engineering hiring is favorable. | |
| Audience Q&A: Security Features and Government Adoption | 1 | 5 | 1 | 0 | Audience members ask about security features for defense contracts and handling semi-structured vs unstructured data. Dwight re-defines unstructured data to semi-structured and shares client examples like Telefonica. | |
| Audience Q&A: Aggregation Improvements and Hadoop Integration | 0 | 5 | 1 | 0 | An audience member asks about batch aggregation and Hadoop integration. Dwight explains native MapReduce vs the aggregation framework and distinguishes Hadoop engine compatibility from HBase competition. | |
| Audience Q&A: Monetizing Open Source and Subscription Conversion | 0 | 5 | 0 | 0 | An audience question asks how to optimize free-to-paid conversion in open source. Dwight delineates lead generation challenges from buyer willingness to pay for mission-critical support and subscriber features. | |
| Audience Q&A: Google Spanner and Distributed Transactions | 2 | 6 | 1 | 1 | When an audience member asks about Google Spanner, Matt steps in to ask for a public definition before Dwight explains the fundamental trade-offs between distributed ACID transactions and horizontal scale. |
Statements from this episode (15)
Turck: NYC had only two billion-dollar exits in a decade, both DoubleClick
“New York has had two billion dollar exits in the last 10 years, and they were both double-click, right?”
Merriman: NoSQL databases omit distributed joins to enable horizontal scaling
“Distributed joins is a hard problem, so the solution that's been chosen by both us with Mongo and the other NoSQL products is to say, you know, we're not going to do joins”
Merriman: Developers will use a few specialized database tools within 5-10 years
“My belief, I'm a little biased, but my belief is that if we kind of project out Say five years, 10 years, is that what we're gonna see is you'll have a few tools in your toolbox for databases.”
Merriman: JSON document models will emerge as the standard NoSQL data model
“My belief, which is opinion, is that in, in the end of these NoSQL data models, that the document-oriented model will be the one that kind of converges on for standard, and specifically JSON style, or JSON based.”
Merriman: MongoDB 2.2 eliminates global read-write lock to improve concurrency
“Historically in Mongo, sort of the concurrency in a single MongoD process on a single server has been suboptimal. And, ah, between servers it's been good, but inside a single server it needed a lot of work. So there was a global reader-irelock, which has been …”
Merriman: Open source causes revenue compression relative to proprietary software
“You are going to get some revenue compression, right, by being open source versus not open source if you have the same number of users, right?”
Merriman: Startup founders are underweighting B2B enterprise opportunities
“He's right. So I think there is an underweighting of activity on B to B because there's plenty of opportunity there.”
Merriman: 10gen finds engineering hiring easier in NYC than Palo Alto
“I'm actually finding right now, because we're hiring engineers both locations, that we're having a little more luck hiring engineers in New York.”
Merriman: In-Q-Tel recently invested in 10gen
“We announced about a week ago that In-Q-Tel invested in TenGen, right?”
Merriman: HBase is more directly competitive with MongoDB than Hadoop
“I think Hadoop, or HBase, which is a Hadoop subproject, that's more of a, that's something that's more of an alternative or competitive with Mongo, where you would look at A versus B”
Merriman: SNMP support is MongoDB's only non-free subscriber feature currently
“Like, I think the only thing in the subscriber edition right now that, that, that is non-free is SNMP support, for example.”
Merriman: Fast distributed transactions across hundreds of commodity servers are impossible
“If we have a thousand server cluster running on commodity hardware, on a commodity network, and you want to touch data and mutate it that's on 600 servers, and then you want to commit that all or nothing with isolation and option to roll it back, and you want …”
Merriman: MongoDB only supports single-document ACID transactions
“What we do currently in Mongo is we do these little microtransactions, which sort of, say, give you ACID properties, but only on a single document at a time, right? Because a single JSON document lives on one and only shard in a Mongo cluster at a given point …”
Merriman: MongoDB's name is derived from the word 'humongous'
“Right, so the name is the middle of the word humongous, so it was kind of a play on this concept of scale, right, so that's where the name came from.”
Merriman: Modifying Postgres wouldn't deliver 10gen's desired architecture
“We looked at the traditional stuff and we thought, well, you know, could we take something, can we take Postgres and start changing, you know, changing code and make something we like? And we felt like we would never, Land where we wanted to get to if we did t…”