Kafka

“All of our events flowing through Kafka, they've been populated into Cassandra, which then we run Spark instances that kind of model on top of the data.”

“In fact, banks is are definitely leading the way in terms of putting Kafka To you know, very sort of ambitious applications.”

“Fundamentally, companies want to collect all sorts of data, and there isn't just a database and a warehouse anymore. There are lots and lots of distributed systems, which means that we need to move to a sort of platform-centric approach, and this will, this is…”

“You know, Kafka powers more than, ah, 1.2 trillion messages, ah, written per day. It is, ah, it powers more than 3.4 trillion, ah, messages, ah, read per day. All that amounts to more than one petabyte of streaming data. And that is across thousands of produce…”

“Since we open sourced it, you know, roughly five years ago, Kafka is used in thousands of companies worldwide, from Uber, and LinkedIn, and Netflix, all the way to traditional enterprises like eBay, and PayPal, and Cisco, and Goldman Sachs.”

“Kafka was, in general, like, a nice way where people used to pipe the results of, like, their Spark jobs. But often cases, what they do is, like, they hit Kafka hard and bring Kafka down because they're trying to, like, actually send data from, like, hundred e…”

“One of our clients is one of the largest social media companies in the world, and we actually pull in streaming data via Kafka.”

“So the scale that we're operating at, we're pushing about a hundred megabytes per second through Kafka.”

“Like, I don't think the popularity of Kafka means that, that, that databases, you know, go away.”

“So, you know, I think we're ending up leaning heavily into being a Kafka shop, building more of the stuff into base plate, into more of these core services”

“It has been to, in my opinion, a key enabler of microservices. I think it's pretty difficult to build and operate a decentralized set of microservices without first adopting something like Kafka in your organization to just move the data between all of these v…”

“Things like Kafka eventually came out of LinkedIn that was based on work that we had been doing at Yahoo”

“It essentially gives you the ability to introspect streams, and it's super, super granular, super detailed, and it gives you know, better, better information than what you would get natively from Kafka.”

“So Confluent which is the company behind Kafka which is like the real-time sort of message bus, like move, move data around in real-time had a wonderful IPO. They just checked today. There are like seventeen billion market cap.”

“In the past, when we were doing data platform, there was no Kafka, for example, we created a Kafka-like system ourselves, but we are lucky that, yeah, we created an HBase, sort of HHive-like system ourselves with hundred petabytes of size of data, right?”

“And that's an architecture I would say that we're seeing a lot of out there is Kafka as opposed to more traditional batch-oriented ETL.”