Sep 16, 2021 · 42m · we-live-to-build
Why Vague Data Searches Produce Vague Results — and Who Profits
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of the We Live to Build podcast, host Sean Weisbrot interviews Nomad Data CEO Brad Schneider to explore how artificial intelligence and natural language processing revolutionize commercial data discovery. Their conversation delves into the alternative data landscape, the challenge of establishing ground truth amidst bias, and the critical need for safety guardrails in autonomous AI systems.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Sean holds 28.3% of the talking time here. How this is scored →
Sean as informed peer 4.4 Guest teaching 5.9 Guest disagreement 2.4 Sean pushing back 2.4
speaking balance: gold is Sean, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 40:16 Brad rejects sci-fi premise of self-rewriting omnipotent AI
Brad directly rejects Sean's suggestion that an advanced AI could magically bypass system limits, noting that no matter how smart an AI is, it cannot defy the laws of physics.
Hardest push from Sean ▶ 40:16 Sean challenges Brad on autonomous self-rewriting capabilitiesSean refuses Brad's architectural safety defense and presses whether a sufficiently intelligent AI could discover its own flaws and rewrite its code regardless of permissions.
Biggest teaching moment ▶ 21:27 Brad educates Sean on ground truth limits and training biasBrad counters Sean's assumption that AI eliminates bias by explaining how machine learning models ingest human bias and fail without reliable ground truth verification.
Sean holds their own ▶ 22:44 Sean leverages personal China experience on data veracitySean demonstrates domain credibility by citing his decade living in China to assert that official governmental GDP metrics cannot be taken at face value.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Sean as informed peer | Guest teaching | Guest disagreement | Sean pushing back | Why |
|---|---|---|---|---|---|---|
| The Origins and Mission of Nomad Data | 3 | 5 | 1 | 1 | Sean asks standard introductory questions about Nomad Data's value proposition. Brad provides a clear breakdown of the fragmented alternative commercial data market and his entrepreneurial background. | |
| The Challenges of Navigating Commercial Data | 4 | 6 | 2 | 2 | Sean asks whether platforms like Facebook sell their data or keep it proprietary. Brad explains that tech giants hoard their data as a moat and describes why navigating thousands of external data vendors is like reading a massive diner menu without context. | |
| Blending Human Intelligence with AI for Data Discovery | 5 | 6 | 2 | 2 | Sean references search indexing within his own company to ask about search strategies across database columns. Brad reframes the problem, explaining that database columns describe what data is rather than the cross-disciplinary business use cases it can answer. | |
| The Limits of Human Recall and AI Matchmaking | 4 | 5 | 3 | 3 | Sean suggests machine learning is the only viable path to search commercial data, prompting Brad to nuance the claim by outlining filtering and tagging before explaining how human recall limits necessitate AI matchmaking. | |
| Sources of Alternative Commercial Data | 4 | 6 | 1 | 1 | Sean asks how vendors acquire data to sell commercially. Brad educates Sean on multiple data generation channels, including web scraping, business exhaust data, Freedom of Information Act filings, and municipal building permits. | |
| Digital Footprints, Aggregated Data, and Targeted Marketing | 4 | 6 | 2 | 2 | Sean asks about the efficacy of VPNs and privacy tools, sharing a targeted advertising anecdote. Brad explains that most commercial monetization relies on high-level data aggregation rather than deanonymized personal dossiers. | |
| User Data Monetization and Participant Selection Bias | 5 | 7 | 2 | 2 | Sean raises blockchain projects proposing to pay users for their data. Brad explains why user-paid panels suffer from severe selection bias, prompting Sean to connect this dynamic to psychological research flaws and historical scientific lobbying. | |
| The Ground Truth Challenge and Macro Data Verification | 6 | 7 | 3 | 3 | When Sean posits that AI can eliminate human bias, Brad corrects him by noting training data inherits human biases and introduces the concept of ground truth data. Sean draws on a decade of living in China to validate challenges in relying on official figures. | |
| Satellite Imagery and Unalterable Physical Data Sources | 5 | 6 | 2 | 3 | Sean asks if foreign governments can manipulate port and shipping figures. Brad explains how alternative data sources like transponder AIS beacons and satellite constellation imagery circumvent governmental manipulation. | |
| Big Tech Ecosystems and Data Privacy Restrictions | 4 | 6 | 2 | 2 | Sean asks if corporate data dominance will remain unchanged for decades. Brad shifts focus to privacy crackdowns by ecosystem owners like Apple, and demonstrates how high-frequency alternative data detected the COVID economic rebound before mainstream news. | |
| Data Quality Versus Volume and Ground Truth Limits | 5 | 7 | 3 | 2 | Sean asks if data volume will overwhelm AI architectures. Brad refutes this using compute economics, explaining that the real bottleneck is data quality and missing context rather than raw scale, citing genomics and protein folding. | |
| The Black Box Dilemma and Trusting Advanced AI | 5 | 6 | 4 | 4 | Sean questions how humans can trust machine learning models if they are opaque black boxes. Brad counters with a philosophical analogy, arguing humans routinely trust other people despite completely lacking visibility into their underlying neural firing. | |
| Humanoid Robots and AI Safety Guardrails | 5 | 6 | 5 | 5 | Sean raises concerns about autonomous systems taking over and AI rewriting its own code. Brad firmly grounds the debate, arguing architectural guardrails and physical impossibilities prevent runaway AI scenarios. | |
| The Future of Data Ecosystems and Episode Conclusion | 3 | 4 | 1 | 1 | Sean asks for closing thoughts and contact details. Brad summarizes the virtuous cycle between growing data availability and AI applications before Sean concludes the episode. |
Statements from this episode (19)
Schneider: Thousands of Companies Currently Sell Commercial Data
“So just to take a step back, there are literally thousands of companies today that sell data.”
Schneider: Big tech companies do not sell proprietary user data
“The big guys are keeping it to themselves. That is their competitive advantage.”
Schneider: 50 to 100 data vendors sell Twitter sentiment data
“There are literally, you know, 50 to a hundred that I know of data providers that just sell sentiment based on, on Twitter data.”
Schneider: Column headers and data formats cannot reveal dataset use cases
“A lot of people try to characterize data by what columns it has, what format those columns are in, what's in the actual cell in a given database table. But that doesn't really tell you what the data can do. That, that more tells you what the data is.”
Schneider: US shipping container import filings are digitized and publicly purchasable
“So any container that comes into the United States, there's form filing requirements that have to be done. All those forms are digitized under the, I believe it's the Freedom for Information Act, and they're publicly available. Granted, you have to buy them fr…”
Schneider: Most commercially sold data is aggregated, not personally identifiable
“You know, I think people can rest assured that most, I mean, I've been in this space for probably 10 years at this point, and the kind of information being sold is mostly not at a, Personal level. It's been highly aggregated.”
Schneider: Paying Users for Data Is Infeasible Without Detailed Profiles
“I hate to say never, and I hate to make long-term predictions because I've been wrong in many cases, but from where I'm sitting, it just doesn't really seem feasible unless you're providing very rich information about yourself.”
Schneider: Paid User Panels Suffer Inherent Selection Bias
“The problem is whenever you recruit people for anything, you introduce a bias, right? If you are paying people to do something, you're enticing people that that amount of money is worthwhile. Those people don't mind giving up information about themselves. And …”
Weisbrot: Chinese government data can never be trusted
“Having lived in China for 10 years, I think it's safe to say it's impossible to trust any source or any piece of data that comes out of the Chinese government ever.”
Schneider: Governments cannot easily manipulate global AIS maritime tracking data
“Then you have forms of information such as AIS data. So every ship around the globe has, you know, think of it as a collision warning system, a ship to ship communication system. And these are reporting the locations of the ship, the name of the ship, and this…”
Schneider: Governments cannot feasibly falsify physical activity under satellite imagery
“Very hard for the Chinese government or for any government to manipulate those images Over a long period of time, right? You have to know exactly when the satellites are going to fly overhead. People are changing which constellations of satellites they use. Th…”
Schneider: App developers monetized cross-app visibility by selling installation data
“Many app companies start to sell that as a data source where they'll report the number of installations for a given app across millions and millions of phones.”
Schneider: COVID mortality uncertainty proves data bias matters more than volume
“It's like we still don't definitively know what the mortality rate from COVID is. There's tons and tons of data, but it all has very different biases, so it's not a, it's not something that's easily solvable with more data. It's more about having the right dat…”
Schneider: Genomic data alone cannot predict disease without missing lifestyle data
“You know, services like 23 and Me are tracking complete breakdown of Every single base pair in a human body. But the problem is we don't have every single one of their medical histories. We don't know everything they ate every single day of their lives. We don…”
Schneider: Personalized medicine via AI will bear fruit in 10-20 years
“I think we'll start seeing fruit born probably in the next kind of 10 to 20 years.”
Schneider: We may never deeply understand the inner workings of ML models
“I don't know that we can come to much better of an understanding about what's going on in the middle or what that means, because there's so many little things happening.”
Schneider: AI trust will rely on observed history, like human relationships
“You just don't understand it, and you need to build up that shared history together to understand, is this something to be trusted? Is this something to be feared? And just like a I mean, a machine learning model, you're gonna see different things that you did…”
Schneider: Centralized traffic coordination is vastly superior to independent human drivers
“You have so many people making independent decisions, which is massively inferior to one thing guiding all traffic, right?”
Schneider: Physical architectural limits reliably prevent rogue AI behavior
“Make an AI as smart as you want, does that mean that it can figure out, again, how to defy gravity, how to travel through time? Some of these things are just physical challenges. You can't do something that isn't physically possible. So if you design a system …”