StarZero All shows Try the podcast clipper →

Feb 21, 2016 · 21m · mad

Large Scale Decision Support Systems // Satya Ramachandran, Neustar (Hosted by FirstMark Capital)

Satya Ramachandran · 17m spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

In this DataDrivenNYC presentation, Satya Ramachandran, VP of Engineering at Neustar, explains the architectural principles, performance evolution, and practical enterprise applications behind building large-scale predictive decision support systems.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 4.4% of the talking time here. How this is scored →

Matt as informed peer 0.7 Guest teaching 2.3 Guest disagreement 0.5 Matt pushing back 0.3
05100:0010:0020:000:10–2:12 · Matt as informed peer 0/10 Defining Decision Support Systems in Big Data Satya opens with a solo presentation introducing decision support systems and MarketShare background. As this is a pure presentation monologue without host participation, host-side scores are zero.2:12–5:53 · Matt as informed peer 0/10 Model-Based Decision Support Systems Architecture Satya continues his monologue describing business workflow challenges between marketers and backend data science modelers. Host scores remain zero due to host absence.5:53–8:47 · Matt as informed peer 0/10 The Data Modeling and Matrix Calculation Process Satya breaks down matrix calculations, feature sets, and coefficient estimation. He mildly critiques vendor fragmentation surrounding proprietary data frames.8:47–10:56 · Matt as informed peer 0/10 Real-World Schema Complexity and User Scenario Analysis Satya presents a real-world customer schema involving 131 million combinations to demonstrate analytical complexity. Host scores are zero during this monologue section.10:56–14:05 · Matt as informed peer 0/10 Exponential Performance Improvements and Deployment Efficiency Satya outlines scale improvements from 2010 to 2015 and introduces his core infrastructure software stack. The segment is entirely monologic.14:05–21:17 · Matt as informed peer 4/10 Key Platform Focus Areas and Presentation Conclusion Matt Turck initiates Q&A by demonstrating strong ecosystem knowledge, referencing specific tech partners like Altiscale and Alation. He asks probing questions about architecture failure points, leading to collaborative discussion.0:10–2:12 · Guest teaching 2/10 Defining Decision Support Systems in Big Data Satya opens with a solo presentation introducing decision support systems and MarketShare background. As this is a pure presentation monologue without host participation, host-side scores are zero.2:12–5:53 · Guest teaching 2/10 Model-Based Decision Support Systems Architecture Satya continues his monologue describing business workflow challenges between marketers and backend data science modelers. Host scores remain zero due to host absence.5:53–8:47 · Guest teaching 3/10 The Data Modeling and Matrix Calculation Process Satya breaks down matrix calculations, feature sets, and coefficient estimation. He mildly critiques vendor fragmentation surrounding proprietary data frames.8:47–10:56 · Guest teaching 2/10 Real-World Schema Complexity and User Scenario Analysis Satya presents a real-world customer schema involving 131 million combinations to demonstrate analytical complexity. Host scores are zero during this monologue section.10:56–14:05 · Guest teaching 2/10 Exponential Performance Improvements and Deployment Efficiency Satya outlines scale improvements from 2010 to 2015 and introduces his core infrastructure software stack. The segment is entirely monologic.14:05–21:17 · Guest teaching 3/10 Key Platform Focus Areas and Presentation Conclusion Matt Turck initiates Q&A by demonstrating strong ecosystem knowledge, referencing specific tech partners like Altiscale and Alation. He asks probing questions about architecture failure points, leading to collaborative discussion.0:10–2:12 · Guest disagreement 1/10 Defining Decision Support Systems in Big Data Satya opens with a solo presentation introducing decision support systems and MarketShare background. As this is a pure presentation monologue without host participation, host-side scores are zero.2:12–5:53 · Guest disagreement 0/10 Model-Based Decision Support Systems Architecture Satya continues his monologue describing business workflow challenges between marketers and backend data science modelers. Host scores remain zero due to host absence.5:53–8:47 · Guest disagreement 1/10 The Data Modeling and Matrix Calculation Process Satya breaks down matrix calculations, feature sets, and coefficient estimation. He mildly critiques vendor fragmentation surrounding proprietary data frames.8:47–10:56 · Guest disagreement 0/10 Real-World Schema Complexity and User Scenario Analysis Satya presents a real-world customer schema involving 131 million combinations to demonstrate analytical complexity. Host scores are zero during this monologue section.10:56–14:05 · Guest disagreement 0/10 Exponential Performance Improvements and Deployment Efficiency Satya outlines scale improvements from 2010 to 2015 and introduces his core infrastructure software stack. The segment is entirely monologic.14:05–21:17 · Guest disagreement 1/10 Key Platform Focus Areas and Presentation Conclusion Matt Turck initiates Q&A by demonstrating strong ecosystem knowledge, referencing specific tech partners like Altiscale and Alation. He asks probing questions about architecture failure points, leading to collaborative discussion.0:10–2:12 · Matt pushing back 0/10 Defining Decision Support Systems in Big Data Satya opens with a solo presentation introducing decision support systems and MarketShare background. As this is a pure presentation monologue without host participation, host-side scores are zero.2:12–5:53 · Matt pushing back 0/10 Model-Based Decision Support Systems Architecture Satya continues his monologue describing business workflow challenges between marketers and backend data science modelers. Host scores remain zero due to host absence.5:53–8:47 · Matt pushing back 0/10 The Data Modeling and Matrix Calculation Process Satya breaks down matrix calculations, feature sets, and coefficient estimation. He mildly critiques vendor fragmentation surrounding proprietary data frames.8:47–10:56 · Matt pushing back 0/10 Real-World Schema Complexity and User Scenario Analysis Satya presents a real-world customer schema involving 131 million combinations to demonstrate analytical complexity. Host scores are zero during this monologue section.10:56–14:05 · Matt pushing back 0/10 Exponential Performance Improvements and Deployment Efficiency Satya outlines scale improvements from 2010 to 2015 and introduces his core infrastructure software stack. The segment is entirely monologic.14:05–21:17 · Matt pushing back 2/10 Key Platform Focus Areas and Presentation Conclusion Matt Turck initiates Q&A by demonstrating strong ecosystem knowledge, referencing specific tech partners like Altiscale and Alation. He asks probing questions about architecture failure points, leading to collaborative discussion.

speaking balance: gold is Matt, purple is the guest (3 minute bins)

0:00 · Matt 0% · guest 100%0:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%15:00 · Matt 29.4% · guest 70.6%15:00 · Matt 29.4% · guest 70.6%18:00 · Matt 2.5% · guest 97.5%18:00 · Matt 2.5% · guest 97.5%21:00 · Matt 6.1% · guest 93.9%21:00 · Matt 6.1% · guest 93.9%
Sharpest disagreement ▶ 6:15 Guest criticizes proprietary vendor data frames

Satya explicitly voices frustration with the proliferation of vendor-specific data frames in predictive analytics, calling out unnecessary fragmentation in flat data structures.

 Hardest push from Matt ▶ 16:20 Host presses guest on architecture failures and lessons learned

Matt Turck pushes beyond the presentation's success narrative by directly asking what broke and failed before Satya achieved his current system performance.

 Biggest teaching moment ▶ 16:43 Guest explains data management as the primary point of failure

Satya educates the host and audience on how analytics platforms break at data management rather than modeling, explaining why strict configuration-driven systems are necessary.

 Matt holds his own ▶ 15:59 Host highlights intimate familiarity with big data vendor stack

Matt Turck demonstrates domain expertise by identifying multiple big data vendors cited in the presentation and connecting them to FirstMark's speaker network.
the scores for every segment, with the reasoning behind each
ChapterTopicMatt as informed peerGuest teachingGuest disagreementMatt pushing backWhy
Defining Decision Support Systems in Big Data 0210 Satya opens with a solo presentation introducing decision support systems and MarketShare background. As this is a pure presentation monologue without host participation, host-side scores are zero.
Model-Based Decision Support Systems Architecture 0200 Satya continues his monologue describing business workflow challenges between marketers and backend data science modelers. Host scores remain zero due to host absence.
The Data Modeling and Matrix Calculation Process 0310 Satya breaks down matrix calculations, feature sets, and coefficient estimation. He mildly critiques vendor fragmentation surrounding proprietary data frames.
Real-World Schema Complexity and User Scenario Analysis 0200 Satya presents a real-world customer schema involving 131 million combinations to demonstrate analytical complexity. Host scores are zero during this monologue section.
Exponential Performance Improvements and Deployment Efficiency 0200 Satya outlines scale improvements from 2010 to 2015 and introduces his core infrastructure software stack. The segment is entirely monologic.
Key Platform Focus Areas and Presentation Conclusion 4312 Matt Turck initiates Q&A by demonstrating strong ecosystem knowledge, referencing specific tech partners like Altiscale and Alation. He asks probing questions about architecture failure points, leading to collaborative discussion.

Statements from this episode (9)

Assertion Partly supported
Neustar acquired MarketShare for $500 million in 2015
“About a couple of months back, we were acquired by Newstar for half a billion dollars.”
Satya Ramachandran Feb 21, 2016 ▶ 2:05
Assertion Not checkable as stated
Enterprise decision support systems still require heavy back-and-forth with modelers
“Today, even the best systems, you see, this involves a lot of back and forth between the business person and actually the back-end folks, which are the modelers and data scientists, right?”
Satya Ramachandran Feb 21, 2016 ▶ 3:24
Assertion Not checkable as stated
The analytics industry remains mostly limited to simple reporting
“We are still working on just doing reporting, pure reporting, or very simple, simple analysis as opposed to analysis like this.”
Satya Ramachandran Feb 21, 2016 ▶ 10:43
Assertion Not checkable as stated
MarketShare ran sub-second simulations on 4.5 billion data points in 2015
“And then in 2015 we worked with online data, about 4.5 billion data points Close to six hundred million possible variables in our model, and we can run these simulations in sub seconds.”
Satya Ramachandran Feb 21, 2016 ▶ 11:18
Insight
Engineering teams should never build custom technology that already exists
“One philosophy we have, ah, for the past year, ah, first five years building the system, what we have learned is to never build what's already there.”
Satya Ramachandran Feb 21, 2016 ▶ 12:37
Assertion Not checkable as stated
Most current business intelligence tools lack a real-time simulation engine
“And the second big focus, and this is where most of the current BI tools lack, is to provide that real-time simulation engine.”
Satya Ramachandran Feb 21, 2016 ▶ 14:33
Insight
Systems break at data management when attempting to productize analytics
“The most important lesson that we had learned in productizing, in analytics thing like that, is to have control on data management. I mean, data management is where it kind of breaks. As long as you have control on data management, the modeling piece becomes a…”
Satya Ramachandran Feb 21, 2016 ▶ 16:55
Assertion Not checkable as stated
Neustar served as the exclusive dynamic pricing agent for Ticketmaster
“So, so we're also doing price. So we basically, we have a pricing tool, also. Right now it's exclusively with Ticketmaster, where we are the dynamic pricing agent, we have Ticketmaster”
Satya Ramachandran Feb 21, 2016 ▶ 18:33
Assertion Not checkable as stated
Neustar maintained data partnerships with Facebook and Twitter for predictive analytics
“We have a partnership with Facebook. We have a partnership with Twitter also. We use their data, ah, to be able to run our analysis as such.”
Satya Ramachandran Feb 21, 2016 ▶ 19:59
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more