ADP uses Databricks for its data layer and Snowflake for analytics workloads
Jack Berkowitz · Leveraging People Data at Scale | ADP Chief Data Officer, Jack Berkowitz · Nov 17, 2022 · at 19:14
Jack Berkowitz, Chief Data Officer at ADP, explains how ADP structures its cloud data infrastructure using both Databricks and Snowflake.
0:00 / 0:09exact quote · 9.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We use Databricks as our, as that data layer, and we use Snowflake to do very specialized analytical database workloads that it's very, very good at.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jack Berkowitz
ADP aims to measure and reveal AI bias rather than eliminate it
“And what we said was, is that we don't believe that bias doesn't exist. In fact, we believe it does exist. And so our role isn't to mitigate all bias, but our role is to at least measure it, understand it, and express it, and make sure that people understand i…”
ADP operates large language models in full production
“We're in full production on top of some of those large, large language models.”
Jack Berkowitz says ADP moved $3.2 trillion in money transactions last year
“So last year we moved about 3.2 trillion dollars.”
ADP uses ML to reduce 21 million job titles into 7,000 categories
“So in any given month, right, those forty million people, there's about twenty one million job titles. We run some pretty sophisticated, ah, machine learning to boil that down to about 6000, ah, or now 7000 key job categories.”
ADP pays roughly 20 percent of all workers in the United States
“We pay in the US, you know, something like 18 or 20% of people, and then we do the taxes on another You know, similar size.”
ADP rejected federated queries to maintain a centralized analytics repository
“The piece we didn't bring from the data mesh was the notion of federated query. Because the problem with federated query is the latency of the query is the worst performing member of the federated group. Right? And so we still have a centralized data repositor…”