StarZero All shows Try the podcast clipper →

Dec 19, 2017 · 23m · mad

Surveillance Platform for Banks // Mayur Thakur, Goldman Sachs (FirstMark's Data Driven)

Mayur Thakur · 19m spoken Matt Turck · 20s spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

Mayur Thakur, Managing Director at Goldman Sachs, presents how advanced computer science algorithms—ranging from temporal joins and search engines to graph analytics and time series spike detection—are deployed in production to monitor billions of daily transactions and prevent financial misconduct.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 1.5% of the talking time here. How this is scored →

Matt as informed peer 0.3 Guest teaching 3.3 Guest disagreement 0.2 Matt pushing back 0.2
05100:0010:0020:001:22–3:25 · Matt as informed peer 0/10 Posing Four Core Computer Science Challenges The segment is a monologue presentation by Mayur Thakur introducing core computer science problems to the audience. The host is inactive, keeping host metrics at 0.3:25–5:53 · Matt as informed peer 0/10 CS Problem 1 - Huge Temporal Joins Mayur details large-scale temporal joins across massive dataset sizes. The host does not speak, maintaining host score floors.5:53–8:23 · Matt as informed peer 0/10 Connecting Computer Science Problems to Compliance Surveillance Mayur explains how compliance surveillance models operate and defines market spoofing. The host remains entirely passive during the presentation.8:23–11:11 · Matt as informed peer 0/10 Compliance Application - Temporal Joins for Market Surveillance Mayur presents real-world compliance applications including search engines for billions of internal communication documents. Host engagement remains zero.11:11–13:16 · Matt as informed peer 0/10 Compliance Application - Graph Analysis for Information Leakage The guest outlines graph analysis used for identifying information leakage across 60 million nodes. Host does not intervene.13:16–18:12 · Matt as informed peer 2/10 Surveillance Platform Architecture & Data Landscape Synthesis Host Matt Turck steps in to prompt Mayur on the underlying technology stack. Mayur details their T+1 Hadoop and MapReduce architecture in response.1:22–3:25 · Guest teaching 2/10 Posing Four Core Computer Science Challenges The segment is a monologue presentation by Mayur Thakur introducing core computer science problems to the audience. The host is inactive, keeping host metrics at 0.3:25–5:53 · Guest teaching 3/10 CS Problem 1 - Huge Temporal Joins Mayur details large-scale temporal joins across massive dataset sizes. The host does not speak, maintaining host score floors.5:53–8:23 · Guest teaching 4/10 Connecting Computer Science Problems to Compliance Surveillance Mayur explains how compliance surveillance models operate and defines market spoofing. The host remains entirely passive during the presentation.8:23–11:11 · Guest teaching 4/10 Compliance Application - Temporal Joins for Market Surveillance Mayur presents real-world compliance applications including search engines for billions of internal communication documents. Host engagement remains zero.11:11–13:16 · Guest teaching 3/10 Compliance Application - Graph Analysis for Information Leakage The guest outlines graph analysis used for identifying information leakage across 60 million nodes. Host does not intervene.13:16–18:12 · Guest teaching 4/10 Surveillance Platform Architecture & Data Landscape Synthesis Host Matt Turck steps in to prompt Mayur on the underlying technology stack. Mayur details their T+1 Hadoop and MapReduce architecture in response.1:22–3:25 · Guest disagreement 0/10 Posing Four Core Computer Science Challenges The segment is a monologue presentation by Mayur Thakur introducing core computer science problems to the audience. The host is inactive, keeping host metrics at 0.3:25–5:53 · Guest disagreement 0/10 CS Problem 1 - Huge Temporal Joins Mayur details large-scale temporal joins across massive dataset sizes. The host does not speak, maintaining host score floors.5:53–8:23 · Guest disagreement 0/10 Connecting Computer Science Problems to Compliance Surveillance Mayur explains how compliance surveillance models operate and defines market spoofing. The host remains entirely passive during the presentation.8:23–11:11 · Guest disagreement 0/10 Compliance Application - Temporal Joins for Market Surveillance Mayur presents real-world compliance applications including search engines for billions of internal communication documents. Host engagement remains zero.11:11–13:16 · Guest disagreement 0/10 Compliance Application - Graph Analysis for Information Leakage The guest outlines graph analysis used for identifying information leakage across 60 million nodes. Host does not intervene.13:16–18:12 · Guest disagreement 1/10 Surveillance Platform Architecture & Data Landscape Synthesis Host Matt Turck steps in to prompt Mayur on the underlying technology stack. Mayur details their T+1 Hadoop and MapReduce architecture in response.1:22–3:25 · Matt pushing back 0/10 Posing Four Core Computer Science Challenges The segment is a monologue presentation by Mayur Thakur introducing core computer science problems to the audience. The host is inactive, keeping host metrics at 0.3:25–5:53 · Matt pushing back 0/10 CS Problem 1 - Huge Temporal Joins Mayur details large-scale temporal joins across massive dataset sizes. The host does not speak, maintaining host score floors.5:53–8:23 · Matt pushing back 0/10 Connecting Computer Science Problems to Compliance Surveillance Mayur explains how compliance surveillance models operate and defines market spoofing. The host remains entirely passive during the presentation.8:23–11:11 · Matt pushing back 0/10 Compliance Application - Temporal Joins for Market Surveillance Mayur presents real-world compliance applications including search engines for billions of internal communication documents. Host engagement remains zero.11:11–13:16 · Matt pushing back 0/10 Compliance Application - Graph Analysis for Information Leakage The guest outlines graph analysis used for identifying information leakage across 60 million nodes. Host does not intervene.13:16–18:12 · Matt pushing back 1/10 Surveillance Platform Architecture & Data Landscape Synthesis Host Matt Turck steps in to prompt Mayur on the underlying technology stack. Mayur details their T+1 Hadoop and MapReduce architecture in response.

speaking balance: gold is Matt, purple is the guest (3 minute bins)

0:00 · Matt 0% · guest 100%0:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%15:00 · Matt 7.4% · guest 92.6%15:00 · Matt 7.4% · guest 92.6%18:00 · Matt 2.1% · guest 97.9%18:00 · Matt 2.1% · guest 97.9%21:00 · Matt 3.4% · guest 96.6%21:00 · Matt 3.4% · guest 96.6%
Sharpest disagreement ▶ 20:11 Shutting down question on specific caught incidents

Mayur firmly refuses an audience question about caught algorithmic spoofing, explicitly stating 'You're not going to get me to say that.'

 Hardest push from Matt ▶ 15:22 Steering presentation toward core technology stack

Matt Turck interrupts the monologue wrap-up to direct Mayur specifically toward explaining the technical infrastructure behind the platform.

 Biggest teaching moment ▶ 9:15 Explaining the mathematical scale of naive temporal joins

Mayur illustrates the mathematical impossibility of brute force temporal joins across 100 million execution rows and 1 billion market rows.

 Matt holds his own ▶ 15:22 Host interjection on architecture details

Host Matt Turck intervenes to refocus the high-level talk back onto concrete technical stack details.
the scores for every segment, with the reasoning behind each
ChapterTopicMatt as informed peerGuest teachingGuest disagreementMatt pushing backWhy
Posing Four Core Computer Science Challenges 0200 The segment is a monologue presentation by Mayur Thakur introducing core computer science problems to the audience. The host is inactive, keeping host metrics at 0.
CS Problem 1 - Huge Temporal Joins 0300 Mayur details large-scale temporal joins across massive dataset sizes. The host does not speak, maintaining host score floors.
Connecting Computer Science Problems to Compliance Surveillance 0400 Mayur explains how compliance surveillance models operate and defines market spoofing. The host remains entirely passive during the presentation.
Compliance Application - Temporal Joins for Market Surveillance 0400 Mayur presents real-world compliance applications including search engines for billions of internal communication documents. Host engagement remains zero.
Compliance Application - Graph Analysis for Information Leakage 0300 The guest outlines graph analysis used for identifying information leakage across 60 million nodes. Host does not intervene.
Surveillance Platform Architecture & Data Landscape Synthesis 2411 Host Matt Turck steps in to prompt Mayur on the underlying technology stack. Mayur details their T+1 Hadoop and MapReduce architecture in response.

Statements from this episode (9)

Assertion Not checkable as stated
Goldman Sachs' internal surveillance graph has 100M nodes and 1B edges
“Or in our case, slightly smaller than the Facebook graph, but still about a hundred million nodes and about a billion edges.”
Mayur Thakur Dec 19, 2017 ▶ 5:11
Insight
Detecting market manipulation is essentially a database joining problem
“A problem about manipulating the market comes down to a problem about kind of joining databases, really, right?”
Mayur Thakur Dec 19, 2017 ▶ 9:22
Assertion Not checkable as stated
Goldman Sachs indexes up to 3 billion internal communications for surveillance
“Think of all the emails, all the chats, all the Bloomberg messages that come or go out of Goldman as being the, ah, the data behind it, right? So like two to three billion documents, ah, behind, billion documents behind it.”
Mayur Thakur Dec 19, 2017 ▶ 10:20
Assertion Not checkable as stated
Goldman's internal surveillance search queries billions of documents in sub-second time
“All these things you can enter the query, and within sub-second, get results, ah, where it's actually going through, ah, well, billions of documents, and finding the relevant results.”
Mayur Thakur Dec 19, 2017 ▶ 10:57
Assertion Not checkable as stated
Goldman Sachs processes billions of market data events daily
“We have about a billion pieces of e-comm in a year. We have about, Ah, you know, hundreds of millions of orders in a single day. Billions of market data events in a single day.”
Mayur Thakur Dec 19, 2017 ▶ 13:43
Opinion
Thakur avoids buzzwords like NLP and machine learning in system design
“Notice I did not quite use buzzwords like NLP, or deep learning, or data science, or machine learning, which I could very well have used, but I don't think it's that useful.”
Mayur Thakur Dec 19, 2017 ▶ 14:43
Disclosure
Goldman's compliance analytics rely on Hadoop and MapReduce batch processing
“So other than search, everything I described is batch processing. We use standard Hadoop. We use MapReduce.”
Mayur Thakur Dec 19, 2017 ▶ 16:48
Assertion Not checkable as stated
Every single alert from Goldman Sachs' compliance surveillance is reviewed manually
“Every single alert that we produce is looked by some human.”
Mayur Thakur Dec 19, 2017 ▶ 19:03
Disclosure
Goldman Sachs targets 95% recall and 20% precision for compliance surveillance
“And we generally hold ourselves to, you know, It's a very recall-driven, if people know what that means, but I'll describe quickly what it means. It's like, the, we want to catch a lot of things that we should be catching, like, 95% of that, right? The more th…”
Mayur Thakur Dec 19, 2017 ▶ 22:58
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more