Qubole processes around 500 petabytes of data monthly
Ashish Thusoo · Making Big Data Accessible Using the Cloud // Ashish Thusoo, Qubole [FirstMark's Data Driven] · Nov 9, 2016 · at 15:10
Ashish Thusoo, founder and CEO of Qubole, describes the scale of data processed by Qubole's cloud platform during a presentation at Data Driven NYC.
0:00 / 0:11exact quote · 11.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“You know, for example, we process around 500 petabytes of data every month across workloads ranging from you know, mundane ETL workloads to more complex machine learning workloads to ad hoc analysis and so on and so forth.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Ashish Thusoo
Public cloud does not make financial sense at Facebook scale
“So that's an excellent question, and, ah, there is definitely an, there, ah, you know, there is an inflection point where people, ah, think that, ah, the total cost of ownership, ah, you know, once you're, ah, so suppose you're a Facebook skill, right, of cour…”
Unmodified on-premise Hadoop distributions fail in the cloud beyond 10 nodes
“If you just take a normal Hadoop distro and try to run it in the cloud, the chances are at 10 nodes it'll work fine as you start growing and, you know, as you start growing and growing and growing further. Things will start breaking because, you know, compute …”
Thusoo: Centralized enterprise data teams usually become operational bottlenecks
“More often than not, this, ah, data team eventually becomes a bottleneck, ah, and, ah, you know, it sort of delays, ah, delays the, ah, availability of data and what people can do with that data.”
Static Hadoop clusters in the cloud defeat the purpose of elasticity
“They just run long-running Hadoop clusters, which completely defeat the purpose of You know, how, how you can leverage the cloud to be dynamically adaptable to your workloads and things like that.”