Goldman Sachs' internal surveillance graph has 100M nodes and 1B edges
Mayur Thakur · Surveillance Platform for Banks // Mayur Thakur, Goldman Sachs (FirstMark's Data Driven) · Dec 19, 2017 · at 5:11
Mayur Thakur, Managing Director at Goldman Sachs, presents on financial surveillance architecture at Data Driven NYC.
0:00 / 0:06exact quote · 7.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Or in our case, slightly smaller than the Facebook graph, but still about a hundred million nodes and about a billion edges.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Mayur Thakur
Thakur avoids buzzwords like NLP and machine learning in system design
“Notice I did not quite use buzzwords like NLP, or deep learning, or data science, or machine learning, which I could very well have used, but I don't think it's that useful.”
Goldman Sachs targets 95% recall and 20% precision for compliance surveillance
“And we generally hold ourselves to, you know, It's a very recall-driven, if people know what that means, but I'll describe quickly what it means. It's like, the, we want to catch a lot of things that we should be catching, like, 95% of that, right? The more th…”
Detecting market manipulation is essentially a database joining problem
“A problem about manipulating the market comes down to a problem about kind of joining databases, really, right?”
Every single alert from Goldman Sachs' compliance surveillance is reviewed manually
“Every single alert that we produce is looked by some human.”
Goldman Sachs indexes up to 3 billion internal communications for surveillance
“Think of all the emails, all the chats, all the Bloomberg messages that come or go out of Goldman as being the, ah, the data behind it, right? So like two to three billion documents, ah, behind, billion documents behind it.”