The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 48 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Evans: Giving enterprise employees Copilot or ChatGPT has not worked well
“Step one was give everybody co-pilot, and that didn't work too well, because it's kind of, or Claude or ChatGPT, because it's kind of like being in 1997 and giving everybody a web browser. Okay, this is the future, but it's not, that's not, what was the produc…”
Evans: OpenAI's new ChatGPT work product is a confusing disaster
“I was playing with a new chat GPT work product, which is clearly not that it's a complete disaster. It's chaotic, confusing mess”
Evans: ChatGPT and Claude are thin wrappers over models
“The only actual real thin GPT wrappers are the apps from these companies like chat GPT is a thin GPT wrapper. Claude is a thin GPT wrapper. Like Claude, the website, Claude, the app, is a thin GPT wrapper. It's an input box and an output box, and a little spri…”
Evans: Mainstream users have never heard of Claude despite Silicon Valley hype
“This disconnect between how everyone in the Valley is obsessed with like Claude is three percent better than ChatGPT this week and next week Gemini is overtaking Claude. But then you look at the user numbers. I know users have never heard of Gemini or Claude. …”
Evans: Lack of network effects makes ChatGPT's early lead fragile
“Given that there's no network effects, or don't, or not yet any network effects in LLMs, that feels very fragile. Like, you know, if you remember, you know, I cannot, you know, remember, like, everyone used Yahoo, and then everyone switched to Google, because …”
Evans: No competing AI app has more than 5% of ChatGPT's usage
“Go look at App Store ranking, or Google Trends, or usage data of any kind that you can get, and you see that basically it's ChatGPT, and then none of the others are more than five percent of that.”
Apple is more likely to replace ChatGPT than build a search engine
“It's much more likely, I think, that Apple will shift questions from ChatGPT to its own foundation model than it would shift questions from Google to its own general search engine, I think.”
Apple will maintain Google defaults while routing specialized queries to AI
“Apple will carry on treating Google as the default general search engine. They'll just peel more stuff out into their own foundation model or into chat TPT or into booking.com.”
Evans: Enterprises require dedicated product UI over raw ChatGPT prompts
“Ask yourself why it is it's a lot easier to go to a company and say, here's a slide maker, than to go to a company and say, well, look, ChatGPT is just multimodal, so just ask what you want. Tell it the slides you want. No, you want UI. You want product around…”
Evans: Ad agencies replace 10-person two-week workflows using ChatGPT and Midjourney
“ChatGPT plus mid-journey means two people in a day instead of a team of 10 people in two weeks.”
Evans: Enterprise AI deployment takes three years, not three months
“This stuff does, yes, this stuff happens for the sake of argument, but it does not happen in three months or three, it takes three years. The deployment for that is going to take, yes, the actually turning it into software that's not just ChatGPT.”
Evans: AI summarizing content without referral traffic makes publisher expropriation claims valid
“And it's not, now they actually are using the content and not sending the traffic to the newspaper. And so that whole newspaper argument that you are using our content goes from being dishonest nonsense to being actually true.”
Evans: Generative AI does not destroy search advertising economics
“I mean, this is kind of, you know, like the kind of the, like the first five minute reaction to GPT, 3.5 is this is a catastrophe for search advertising, which you kind of pull back and think, well, it's already not actually, you know, if I get, I can go to ch…”
Evans: Rebuilding ChatGPT from scratch takes 6–12 months and tens of millions
“How long would it take you to make a chat GPT again yourself from zero? And it's like six months to a year and however many tens or hundreds of millions of dollars. So, you know, it's not quite, you know, anyone can do that at home, but it's like, it's also no…”
Evans: Radically new AI UX will reset default consumer search habits
“Presume that the chat GPT will be 10% worse than Google. I mean, maybe not, maybe it'll be 10% better, but the point is consumers will not care anymore. They will, the experience will be so different that you'll kind of reset people's presumptions and people's…”
Evans: Survey data shows Claude holds between 0% and 1% consumer usage
“If you look at the, you know, survey data on usage, it's,
It's ChatGPT, and then it's sort of two-thirds of that to a half of that is Gemini and Meta AI.
Even though Meta AI is a fiasco, Llama IV was a fiasco, the new models haven't launched yet, but consume…”
Evans: AI cascading model pickers represent a product design failure
“All that you have is you have these ridiculous model pickers, which we have to make sure we mention, like the model picker that cascades out. Should I use GPT? Should I, yeah, should I use O-three or three- Well, how the fuck? What the hell kind of a question …”
Evans: Consumers lack obvious reasons to prefer one AI chatbot over another
“The point is, whatever your use case is for ChatGPT, there's not any particularly obvious reason why you'd be using ChatGPT versus Anthropic versus Meta.ai versus Gemini, and certainly not why you'll stick with it over the next, you know, six months or a year,…”
Evans: OpenAI's ChatGPT plugins and custom GPTs have clearly failed
“We also incidentally saw like part of the attempt to make this work with first of all the plugins and then GPTs, neither of which have worked clearly.”
Evans: ChatGPT is a giant job creator for lawyers and consultancies
“ChatGPT is like a giant job creation scheme for lawyers and consultancies. Because the lawyers have to work out who we're suing, and what all the new contracts look like, and the consultancies, and people actually have to go and deploy all of this stuff.”
Evans: Replacing bespoke software with AI prompts requires AGI
“I mean, I think that only works if you have AGI, which is the third point.”
Evans: Benedict Evans has no personal or professional task for ChatGPT
“I personally do not have a task where chat GPT is useful. There are other people who do. I don't.”
Caron-Brown: Visual AI will evolve faster than text AI
“We're probably going to see way more and faster evolution with the visual stuff than we are with the written.”
Benedict Evans: Many impressive ChatGPT use cases are just basic Google searches
“It drives me crazy as people doing say, oh my God, have you seen what chat GPT can do? And I say, yeah, you've just done a Google search. That's a Google search. Google does that.”
Benedict Evans: ChatGPT's chat UI deceptively implies definitive answers
“To me, this is part of the challenge of ChatTPT versus Google is that Google says here are 10 links that might be the answer. Whereas ChatGPT says this is the answer, and it's, this is the answer. But it's not, you could argue that what it should be saying is …”
Evans: Generative AI generates patterns without structural understanding of underlying objects
“It's making a pattern without any structural understanding of what the actual thing actually is.”
Evans: ChatGPT passes the Turing test by producing C-grade answers
“ChatGPT can pass the Turing test. I mean, you may not think it's a very bright person. I mean, this is the thing. I was plugging in my questions. I remember from when I was at history undergraduate, like, yeah, these are all C grade D grade answers. They are a…”
Evans: The Turing test no longer works as an AI benchmark
“ChatGPT is, you know, it is better, it is in, on some axes, it is cleverer than dumb people. And so that, that kind of means the Turing test doesn't work anymore.”
Evans: 33% to 50% tried ChatGPT, but daily active users are 1% to 5%
“And the top line is basically that in a bunch of developed world countries, something between a third to a half of people have already tried ChatGPT.
And so that's a very glass half full kind of statement.
On the other hand, the number of people you've turned …”
Evans: LLMs are rebundling the curation that influencers unbundled from magazines
“Influences unbundling for magazines and TV, but mostly for magazines. And of course, LLM then, you know, ChatGPT goes off and reads the whole web and synthesizes it for you, so it's bundling all the influences back up to the top again”
Evans: ChatGPT traffic dropped over summer 2023 because students went on break
“This is the reason why ChatGPT traffic fell off last summer, because all the kids went home from school.”
Evans: Only 2% to 3% of Americans trust ChatGPT output
“So a third of people haven't heard of it and two or three percent would trust what it says.”
Evans: Generative AI will force marketing agencies off cost-plus pricing
“We're going to have to shift from a cost plus a percentage basis to something else because our customers are not going to pay us to have a bunch of interns sitting around, juniors sitting around stuff that we could do with ChatGPT. And so if we're going to do,…”
Evans: Customers should not be expected to figure out AI's utility
“It's not the customer's job to know what the technology is for. It's not the customer's job to have be given a ChatGPT account and told, there you are, go off and reimagine how your company does its backup of its processes.”
Evans: LLM interfaces resemble Battleship more than document editing
“When you're using LLM, it's a kind of call and response model. You type the prompt, you press go, you wait, And something comes back. I compared it to playing battleships. You know, you call out your move and you find out if you hit anything and then you wait …”
Evans: ChatGPT's fluent prose masks its underlying lack of understanding
“With chat GPT, the lateral language generation is so good that you can't see the model behind it. It isn't as good as the text. Like the prose generation is perfect. The grammar is perfect. So that hides the fact that the understanding behind it isn't quite th…”
Evans: Open-ended AI models create a paradox between capability and discoverability
“Paradox is theoretically it can do anything, but you don't know what you can do. So you want a list. Theoretically you can ask it for whatever you want, but actually there's the five things that you found that work for you.”
Evans: AI users need dedicated GUIs rather than blank text prompts
“And I was going to say ChatGPT has that sort of question, which is there's a whole bunch of stuff that theoretically you could ask it to do, but actually you want user interface and you want buttons and you listed, you want the list of the five previous querie…”
Benedict Evans: Users will consistently bypass probabilistic LLM safety filters
“You can try and stop it, but people will find ways of getting around it because it's not just doing a binary database lookup. So you might have a keyword filter on Napalm and people will come up with a way of describing Napalm without using the word Napalm.”
Evans: ChatGPT breaks consumer internet economics with compute costs per query
“The idea that it actually costs money to press enter on a consumer internet site, like posting a pic, you're actually like a Facebook if it'd cost money to do a post. And you had to wait 30 seconds for it to appear. Well, that's kind of what ChatGPT is now. Li…”
Evans: Demanding citations from LLMs is a category error
“Asking and asking chat GPT for a source. That's not what it is or how it works. Because one answer is the source is the internet. Two or 300,000 things. The second answer is that when you've said, please give me a source, what these systems do is they give you…”
Evans: ChatGPT's natural language presentation creates an illusion of certainty
“But if you go to ChatGPT and you ask it a question, it gives you one answer, and it presents that in natural language, which looks, which creates this impression of certainty. This is this phrase, it's a confident bullshitter. It might have exactly the same de…”
Evans: ChatGPT succeeded due to open access and content moderation
“The difference is ChatGPT, I think there's two specific things about ChatGPT. One of them is that they just made it open and anyone can plug stuff in. And the other, I think is that they put a lot of effort into content moderation. Like there's like a building…”
Evans: Generative AI lacks knowledge, merely reflecting patterns from training data
“It doesn't know that. It know, it is collectively reading back to you the entire internet, or it is collectively reading back to you every single photograph that it's seen, or it is collectively reading back to you like the collective consciousness of everythi…”
Evans: I will not publish writing that ChatGPT could generate
“One of my, the ways I would think about what I'm writing is I've written this. Is this what anybody else would say? Is this just saying what everybody knows or everyone thinks? Well, then I won't publish it. And now I just say, well, is this what ChatGPT would…”
Evans: Pew data shows 18% of US adults have used ChatGPT
“US adults, 18% have used chat GPT ever, and 33% of 18 to 29, up to 43% have used ever.”
Evans: Roughly one-third of Americans under 30 use ChatGPT for work
“Something like a third of Americans aged under 30 have used it for work for some reason.”
Evans: ChatGPT has not fit into personal workflows after nine months
“Nine months in, I'm an early adopter. I do all sorts of stuff. I've been pushing and pushing and pushing ChatGPT. I haven't really found anything that's useful for me. That's not, it's not useful. That's, I haven't found something that fits into my workflows.”