why aren't all 98 resolved? a statement only gets an assessment when the public
record can support or contradict it. opinions and what-ifs never can, and 3 checkable
ones are still open, waiting for their date. predictions held up or didn't;
assertions are supported or contradicted. on every card:
▮▮▮▮▮ certainty ·
▮▮▮▮▮ debate potential. speakers are clickable
Assertion Partly supported
Socher: Chat ads perform up to 100x worse than search ads
“We actually evaluated that they work about 10 to a hundred X worse than search ads and you have. Twice the cost, about.”
Disclosure
Socher: Google deranked You.com across all sites, causing immediate traffic drop
“We just got deranked in all our sites from Google and saw like a drop right away.”
Assertion Partly supported
Socher: Bing and Google copied You.com's AI chat features
“Bing and Google copy, you know, what we have launched last late last year with you chat. And you know, they may have worked on it before, like some people, you know, claim, oh, for, I mean, we've also met the prompt engineering in 2018, but like, they've copie…”
Assertion Partly supported
Socher: Prompt engineering was invented at Salesforce Research
“When we invented prompt engineering, we actually did it at Salesforce research when I was chief scientist there.”
Insight
Socher: Enterprise Is the Killer Application for Large Language Models
“We realized eventually the killer app for large language models and complex answers is an enterprise.”
Prediction Not checkable as stated
Socher: Thin-Layer LLM Providers Will Function Like High-Capex Telcos
“I think LLM companies, especially just the pure thin infrastructure layer of LLMs are going to look, I think, more and more like telcos in the sense that it's high capex, huge, you know, expenditure to build it, especially if you want to build it from scratch.…”
Opinion
Socher: OpenAI Is a Consumer App Company, Not Pure Infrastructure
“That's why I said, if you're in that thin infrastructure layer, and that was an important qualification because OpenAI is a consumer app company.”
Opinion
Socher: Other consumer LLM apps are rounding errors compared to ChatGPT
“Consumers, once you're really famous and you cross that threshold of just like being well known, being the default for a lot of people, all the other LM apps companies are almost rounding errors to ChatGPT.”
Prediction Not checkable as stated
Socher: LLMs will capture search whenever queries are complex
“And so LMs as part of that unbundling wave of Google, LMs will capture whenever you have a more complex question.”
Assertion Not checkable as stated
Socher: Enterprise OpenAI deployments see weekly active usage drop to 6%
“They had to pay a thousand seat licenses for OpenAI, and then six months later, they realize only six percent are actually using them every week.”
Prediction Not checkable as stated
Socher: AI will turn every individual contributor into a manager
“With AI, every person will become a manager, but most people are not used to managing other people or processes.”
Assertion Not checkable as stated
Socher: DeepSeek model training likely cost $100M-$200M, not single-digit millions
“It's also clear that it probably cost them a 102 hundred million dollars, but it's still incredibly cheaper than billions of dollars that we're told it would take to train these kinds of models.”
Prediction Not checkable as stated
Socher: AI will solve major medical problems within ten years
“Yes. That is one of, like, several, like, several chapters in my book are about that.”
Prediction Not checkable as stated
Socher: UBI will deprive people of meaning and worsen societal crisis
“I think the problem is that while most people complain about their jobs. It does give them meaning. It does give them meaning to be a valuable part of society and to have earned something that they can then give to their family, to their kids, and so on. And s…”
Assertion Not checkable as stated
Socher: Entry-Level Tech Jobs Are Increasingly Automatable
“I think the biggest problem and biggest worry I have is that the entry level jobs right now are more and more automatable.”
Insight
Socher: Programming Knowledge Is Required for Effective Vibe Coding
“But if you don't know how to program at all, you're also not going to be as good of a vibe coder.”
Insight
Socher: AI tools lack switching costs without proprietary enterprise data integration
“There is very little switching cost. Same is true for LLMs in the consumer world, right? None of them are that amazing yet. None of them do enough personalization yet. Now, where there is a lot of switching costs is if you have company internal data or you hav…”
Prediction Not checkable as stated
Socher: AI startups trading at 80x-100x ARR without moats face correction
“We're going to see a correction when companies are trading hundred ADX their ARR and they don't have a real moat and like there's the switching cost is close to zero to go to DeepSeek or something else.”
Insight
Socher: AI can solve every problem in any domain that can be simulated
“In AI, anything you can simulate, AI can solve every problem in that domain.”
Prediction Open · timeframe Dec 2027
Socher: Expects to win $1,000 bet that AGI won't arrive by 2027
“We did this bet and in the bet he has to win. I think it ends in 20, 27. So three things have to be true. We have to have a personal robot that cleans the whole house the way my cleaning team does. And it needs to be purchasable, like for reasonable amounts of…”
Prediction Open · timeframe Apr 2030
Socher: OpenAI, Anthropic, and Grok won't generate 1,000x returns
“I personally would just, I also just love investing in early stage where you can have thousand X's and so on. I just don't see a thousand X's for those companies.”
Prediction Didn’t hold up
Socher: An open-source GPT-4 equivalent will arrive by end of 2023
“I predicted that we'll have a GPT-IV equivalent model Before the end of the year, that's open source.”
Insight
Richard Socher: Enterprise Incumbents Hold a Key Advantage in Proprietary Data
“So it's complicated in the sense that unsupervised data, just raw internet text is easily accessible, but there's still a lot of data sets out there that are not out there. They're actually stored in a private databases. And indeed, if you want to answer custo…”
Insight
Richard Socher: AI Startups Can Build Moats Through Distribution and Partnerships
“Turns out you can have moat other than your backend AI model, right? It's distribution, it's partnerships. Your sales funnel processes and so on.”
Assertion Partly supported
Richard Socher: Bing and Google Copied YouChat Months After Its Launch
“We've also seen Bing and Google copy what we have launched late last year with you chat. They've copied us in the sense that we've launched it earlier and then they launched something very similar three to four months after.”
Insight
Richard Socher: Google's Ad Revenue Creates an Innovator's Dilemma Against AI
“That kind of big change will be hard for Google too, because they make five hundred million dollars a day with privacy invading advertisements on that page. And so you don't just willy nilly change most of that page and you get rid of the five, six ads that ar…”
Assertion Supported
Socher: Calling You.com a thin LLM wrapper is completely false
“But it's something that, you know, some VCs don't appreciate and understand the complexity of, and then they say, oh, like u.com is just a thin wrapper around a large language model, which is very far from the truth.”
Prediction Not checkable as stated
Socher: Base LLMs will become commoditized like databases
“And I think it won't matter that much which database you use, just like it won't matter that much, which LM you use, but it matters what you do with it, how you tune it what kind of training data you add onto it to fine tune it.”
Prediction Didn’t hold up
Socher: Open-source GPT-4 equivalent model will launch before end of 2023
“I predicted that we'll have a GBD four equivalent model before the end of the year. That's open source. Of course, GBD four keeps getting better and better. So my prediction was for the version we had like a few months ago,”
Assertion Supported
Socher: Google Earns $500 Million Daily From Main Search Ads
“They make five hundred million dollars a day with privacy invading advertisements on that page.”
Assertion Contradicted
Socher: You.com launched the world's first search-integrated LLM
“We've launched uChap and had the first LM with a search backend and citations and web links and so on in the search context which we launched last December before anyone else in the world”
Prediction Not checkable as stated
Socher: Physical jobs will become expensive bottlenecks constraining AI GDP growth
“The tasks that are physical are getting more and more expensive and they're going to become the new bottlenecks, right? So your carpenter, the people like who built your house all of those kinds of jobs are going to be more and more expensive. And then they're…”
Insight
Socher: Next-token prediction alone does not constitute general intelligence
“I find it hard to call something Artificially super intelligent or generally intelligent. If it, all it does is predict the next tokens and you say, oh, predict this next token. It'll predict that next token. I think an intelligent existence probably needs to …”
Assertion Contradicted
Socher: Commercial incentives prevent developers from building goal-setting AI
“Because companies need to make money and governments want to have a productive economy, no one is working on AI, just doing whatever it wants to do because that doesn't make any money. So no, one's working on AI setting its own goals.”
Opinion
Socher: AI community should focus on current risks over existential sci-fi scenarios
“To have more folks talk to each other about real risks and let, you know, a small set of folks continue to think about existential risks, but not scare people so much with very interesting sci-fi and just general fiction scenarios that would make fun action mo…”
Disclosure
Socher refused to sign Elon Musk's AI pause petition
“I did not sign it. I don't think it makes sense to pause the training of models.”
Assertion Supported
Socher: Peer reviewers publicly rejected the original prompt engineering paper
“We invented prompt engineering which was majorly rejected publicly on open review an idea that made no sense to the reviewers.”
Assertion Not checkable as stated
Socher: LLMs are already good enough for most business tasks
“And so LMs are already good enough. They just need to be brought into companies to be actually made useful.”
Assertion Not checkable as stated
Socher: Object detection in computer vision is practically solved
“For example, object detection and computer vision. It's actually kind of solved. We can classify most objects on the planet.”
Assertion Supported
Socher: OpenAI generates vast majority of revenue from ChatGPT
“They make their revenue, the vast majority of their revenue from a consumer app called ChatGPT.”
Assertion Not checkable as stated
Socher: Anthropic under pressure due to Claude's low consumer market share
“Anthropic has a lot more pressure to keep building the best models because Claude is so much smaller in terms of market share for the consumer app.”
Assertion Not checkable as stated
Socher: 80% of iPhone users never change a single device setting
“80% of all iPhone users never change a single setting of any kind.”
Insight
Socher: Natural language is not the best interface for all AI tasks
“As much as I love natural language is not the single best interface for a lot of different types of answers. Sometimes you want to see a map. Like sometimes you want to see a table. Sometimes you want to see a map with a bunch of specific overlays.”
Insight
Socher: Web action AI agents are in a valley of disillusionment
“We're sort of in this valley of disillusionment on a lot of these, what I call action agents that go on the web and actually do something for you and take actions that you can't undo and say you buy a ticket that's not refundable or something. There's this val…”
Insight
Socher: Google's AI struggles stem from ad-model innovator dilemma, not tech
“I think it's never been a question of technical strength for Google. It's just a question of classic innovator's dilemma. You make money by showing ads and lists or blue links. So it's hard to give people just a straightforward, useful answer.”
Opinion
Socher: Apple has failed to execute effectively on its consumer AI position
“In theory, Apple would be so well positioned, but in practice, they've been not doing much.”
Assertion Not checkable as stated
Socher: DeepSeek bypassed top LLMs rapidly due to low switching costs
“I think one is that personalization, and that's why it's so easy for people to switch around LLMs too, you know, like DeepSeek overtook almost every other thing other than ChatGPT within the weeks.”
Opinion
Socher: The market is not currently in an AI bubble
“I don't think we're in an AI bubble period.”