Spark

“All of our events flowing through Kafka, they've been populated into Cassandra, which then we run Spark instances that kind of model on top of the data.”

“And what's really interesting is Flink is already faster than Spark.”

“They used Tachyon to accelerate Spark jobs from hours to seconds”

“So the one that we've got here in the U.S., it's primarily a Cloudera Hadoop stack that we've used a blueprint that was essentially blessed by our brethren over in French, in France and with that, we've got R and Python and Spark. We use some Dataiku along the…”

“Kafka was, in general, like, a nice way where people used to pipe the results of, like, their Spark jobs. But often cases, what they do is, like, they hit Kafka hard and bring Kafka down because they're trying to, like, actually send data from, like, hundred e…”

“I think there's still more hype around Spark than actual value extraction from it.”

“So probably still today the most common is using kind of hand coding tools so programming languages, Python, Spark SAS and so on.”

“Spark, I think, is being used very, very broadly for advanced analytics, machine learning, in some cases for streaming data, and those scenarios are all very, very complimentary to Snowflake.”

“Anyone who's trying to do high concurrency will usually realize that Spark is not built for high concurrency.”

“Two or three years ago, we set out to re-implement all Spark in C++ in what we call the really, really fast, what's called MPP engine, Massive Apparel Processing Engine.”

“There are actually two orders of magnitude more human beings on the planet that can write SQL than can write Spark or Scala or whatever.”