Ben Johnson: Routing layer is the most valuable part of Timber's pipeline
Ben Johnson · A New Kind of Logging System // Zach Sherman & Ben Johnson, Timber (FirstMark's Data Driven NYC) · Dec 5, 2018 · at 0:16
Ben Johnson is the co-founder of Timber.io. He is introducing a presentation at Data Driven NYC about telemetry data architecture and log routing.
0:00 / 0:10exact quote · 10.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We want to talk to you about what we have found to be the most valuable aspect of our data pipeline. We call it the routing layer. It's really been the foundation of all the value we've extracted out of that.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Ben Johnson
Ben Johnson: Real-time data processing was a major challenge with Elasticsearch
“So being being real-time and processing this data was really important, and that was a real challenge with Elasticsearch.”
Ben Johnson: Timber struggled to scale Elasticsearch past multiple terabytes
“What we found though is that while it was good getting started, we encountered a lot of challenges with it, especially when we started to cross over into the multiple terabyte range. And we had a lot of trouble scaling it.”
Ben Johnson: No single storage engine could completely replace Elasticsearch
“What was interesting is that all of these storages have different strengths, but none of them really could just completely replace Elasticsearch.”
Johnson: Timber intends to open-source its log routing technology
“Our intent is to open source this project.”