May 19, 2025 · 54m · big-technology
Is ChatGPT The Last Website?, Grok’s System Prompt, Meta’s llama Fiasco
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Alex Kantrowitz and Ranjan Roy analyze how conversational AI is displacing traditional web traffic, examine algorithmic bias and system prompt vulnerabilities in Grok, and debate whether frontier model scaling has hit a wall amidst Meta's Llama delays.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Alex holds 58.7% of the talking time here. How this is scored →
Alex as informed peer 7.6 Guest teaching 3.0 Guest disagreement 3.0 Alex pushing back 3.2
speaking balance: gold is Alex, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 36:03 Dismissing the god-model necessity
Ranjan directly challenges Alex's premise that larger models are necessary, arguing that LLMs struggle with basic messy contextual data and that waiting for an all-powerful model is unrealistic.
Hardest push from Alex ▶ 34:11 Refusing the expectations-only explanationAlex firmly rejects Ranjan's framing that Meta's Behemoth delay is just a PR expectation issue, pointing out parallel delays across OpenAI and Anthropic to argue that scaling has hit a wall.
Biggest teaching moment ▶ 27:31 Reframing AI alarmism with algorithmic cynicismRanjan reframes Alex's fear of hidden system prompts shaping user thought by arguing that recommendation algorithms from TikTok and Meta have already normalized this dynamic for years.
Alex holds their own ▶ 39:45 Deploying IBM enterprise study to defend scaling thesisAlex cites precise statistics from an IBM survey of 2,000 CEOs showing that only 25% of AI deployments achieved positive ROI, proving that model capabilities still need significant improvement.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Alex as informed peer | Guest teaching | Guest disagreement | Alex pushing back | Why |
|---|---|---|---|---|---|---|
| The Death of Clicks: SimilarWeb and Cloudflare Scraping Metrics | 8 | 2 | 1 | 1 | Alex anchors the discussion with specific SimilarWeb traffic rankings and detailed crawling metrics from Cloudflare's earnings call. Ranjan collaborates and double-checks the exact crawl-to-visit ratio figures for OpenAI and Anthropic. | |
| Grok's System Prompt Glitch and the Risk of Algorithmic Bias | 8 | 2 | 2 | 2 | Alex provides a clear technical breakdown distinguishing fine-tuning from system prompts and quotes specific prompt instructions from Grok and reporting from The Guardian. Ranjan adds commentary regarding UX incentives for model sycophancy. | |
| The Post-Web Economic Model and Direct Content Ingestion | 7 | 4 | 4 | 4 | Alex presents ideas on direct content ingestion and notes Wikipedia's traffic drop, while expressing concern over hidden system prompt manipulation. Ranjan counters with cynicism, arguing social media algorithmic feeds have already been manipulating users in the same way for years. | |
| Mid-Show Announcements and Google IO Preview | 8 | 4 | 5 | 6 | Alex challenges Ranjan's claim that model delays are purely expectation-driven by citing missed timelines across Meta, OpenAI, and Anthropic, backed by an IBM CEO survey and academic quotes. Ranjan pushes back, arguing that current models are sufficient and that failed adoption stems from poor enterprise implementation. | |
| Enterprise AI Economics and DeepMind's AlphaEvolve | 7 | 3 | 3 | 3 | Alex outlines DeepMind's AlphaEvolve paper, highlighting benchmark data on algorithm discovery and kernel optimization. Ranjan humorously focuses on the kissing number problem while maintaining his skepticism toward model-centric hype. |
Statements from this episode (12)
ChatGPT traffic grew 13% month-over-month while other top websites declined
“If you look at the traffic change month over month, Google, at YouTube, Facebook, Instagram, all going down. ChatGPT up, 13% month over month. Then everything else that follows, X, WhatsApp, Wikipedia, Reddit, Yahoo Japan, All going down. And so, ChatGPT stand…”
Anthropic crawls 6,000 web pages per visitor referred, Cloudflare data shows
“OpenAI, 250 mentions of a site relative to one direct traffic sent to the website. Anthropic, 6000. I mean, 6000 crawls. 6006 thousand crawls to one.”
Hidden system prompts in AI chatbots pose subtle ideological steering risks
“The thing is, what, all these chatbots have a, often hidden, system prompt, and they have an ideology, one way or the other. Sometimes, most of the times, not as overt as this, And that to me is the risk about these things becoming the last website is that you…”
Chatbot consolidation is worse for information health than previous distribution models
“It's one of six or seven, let's call it. It's a real problem. It's a huge problem. It's a, from a pure kind of like information health standpoint, it's far worse than anything we have seen”
Grok's system prompt suffered an unauthorized change on May 14, xAI says
“It was on May 14th at approximately 3:15 a.m. Pacific Standard Time, an unauthorized modification was made to the Grok response. Bots prompt on X.”
ChatGPT surpasses Wikipedia to become the fifth most-visited website globally
“ChatGPT has overtaken Wikipedia. So ChatGPT is site number five, and Wikipedia is eight.”
Wikipedia is effectively finished now that ChatGPT has eclipsed its traffic
“To me, that's basically like Wikipedia is done.”
Meta, OpenAI, and Anthropic are all hitting walls with model scaling
“So, it could be that this idea of scaling to lead to improvements, which we've talked about on the show a couple, for the past couple weeks this is three, meta, open AI, And Anthropic, they all seem to be, ah, running into some bumps on, in their efforts to im…”
AI models cannot solve arbitrary, messy data without domain-specific training
“The idea that all there's going to be models so smart that they will and capable that they can take any kind of input, no matter how disjointed or context specific they are, let's call it. I think like that to me, it's just not going to happen or it, maybe it …”
Current AI models can handle most business processes if implemented correctly
“Most business processes that exist in the world are pretty straightforward and to the models of today can handle them if the implementation's done right.”
Cohere hits $100M ARR, falling short of its $450M investor pitch
“One from Reuters was that Cohere scales to a hundred million in revenue annualizes in May, Seemingly positive, exciting number, but then from the information is that Cohere, that basically they had shown investors they'd be making four hundred fifty million AR…”
Perplexity and PayPal partner to enable native checkout directly within chat
“So perplexity announced a partnership with PayPal. We've talked about this a lot and perplexity has done a lot with shopping and they'll let you ask a question. They'll show you a bunch of potential results. Now with PayPal, you can check out directly, handle …”