Narayan: Streaming will remain niche until tooling matches batch systems
Arjun Narayan · Fireside Chat: Arjun Narayan (Founder & CEO, Materialize) with Matt Turck (Partner, FirstMark) · Mar 15, 2021 · at 21:17
Arjun Narayan is the Founder and CEO of Materialize, a real-time streaming database startup. He discusses adoption hurdles for streaming data architectures.
0:00 / 0:14exact quote · 14.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Until streaming gets to the same level of tooling, and a large part of that tooling is dbt as existing batch systems you know, it will still remain a fairly niche technology.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Arjun Narayan
Narayan: Materialize provides 99.9% of batch database functionality
“And we really think materialize the product is the first database that gives you know, I don't want to say literally all the functionality, but to the morally speaking, you know, 99.9% of the functionality that you can write in a batch database.”
Narayan: Software requiring code rewrites in new languages is largely doomed
“Any pitch. I'm generally very skeptical where you can tell folks, you can have all these great new benefits of low latency or whatever it is, but you got to start all over from scratch, right? You have to throw everything out there and you're going to rebuild …”
Narayan: Timely Dataflow was first stream processor to match batch processing capabilities
“It was sort of the first, what I would describe as The very first stream processor that could do everything that batch processors could do.”
Arjun Narayan: Streaming data processing has expanded beyond niche financial applications
“And what we've been seeing is that streaming has started to become over the, over several decades and particularly in the past few years, much, much more broadly applicable. Beyond those small niche use cases as more and more businesses and use users benefit f…”
Narayan: Apache Kafka is essential for building and operating microservices
“It has been to, in my opinion, a key enabler of microservices. I think it's pretty difficult to build and operate a decentralized set of microservices without first adopting something like Kafka in your organization to just move the data between all of these v…”
Arjun Narayan: Standard analytics databases add latency by lacking incremental computation
“So, so stopping everything and recomputing from scratch is not really a framework that scales to these lower and lower latencies, which is why fundamentally a lot of analytics databases today including some of the more famous ones, they would prefer it if you …”