Web Scraping
topic on 3 shows · 3 statements across 3 episodes
Latent Space 
Top Founders 
the a16z Podcast
3 statements about Web Scraping, every show
Bornstein: AI data access will be solved by scraping, not protocols
“These problems are rarely solved by defining a new protocol and just saying like, Hey, if we make it easy for people to give away their core assets, they'll just do it. You know, obviously, you know, that's very unlikely to work, but someone eventually will so…”
Klein: Web scraping workflows should use a tiered waterfall architecture
“If you're building web scraping workflows, what you should do is have a waterfall. You should have the first request is a curl to the website. See if you can get it without even using a browser. And then the second request may be like a scraping specific API. …”