The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 218 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 4 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Chollet: 50,000x LLM scaling yielded flat progress on ARC benchmark
“Because between, like, GPT-II and GPT-IV. There's been this 50,000 X scale up of base models that has resulted in, in, in basically a flat curve. On Arc.”
Kiela: Core language model development is almost solved and plateauing
“It's not even really about language models anymore. That has almost been solved, right? That's kind of why you see things plateauing off a little bit as well.”
Kiela: Long-context LLMs are inherently incredibly wasteful
“Long context models are inherently incredibly wasteful. You're paying for all this compute, and that's maybe why some of the companies that are trying to really sell long context model, long context window models, they will make more money from that, right?”
Misra: CapCut relies on simple UI and TikTok distribution rather than AI
“Whereas I think Capco's approach is much more traditional. They're actually just trying to do exactly what Canva did, which is, okay, let's make the UI simpler and add a few more buttons and essentially bootstrap it to TikTok and distribute it like crazy.”
Misra: AI video has no positive use cases in documentary footage
“But on the documentation side, there's no good thing that AI video can do. Not one good thing, right? Like there's, it's all bad. There's nothing good.”
Ada uses runtime guardrails to prevent inaccurate AI responses entirely
“Increasingly, we make those assurances in runtime, so it's actually not possible for ADA to deliver an inaccurate response.”
Masad: Traditional AI coding tools do not matter compared to agents
“So all the stuff that we built around coding AI, I don't think it matters. What really matters is being able to chat with an agent and get an application on the other end.”
Masad: Long-tail SaaS products will disappear due to AI software generation
“As, you know, non-professional software engineers are able to build their own software I think that the demand for the, you know, plethora of SaaS that we have today will also go down, and that will reduce, because you'll be able to generate software on the fl…”
Masad: Total software engineer headcount will stay roughly constant
“So I think I think the population of software engineering will probably be constant and not grow all that much.”
Borgman: Tabular acquisition gives Databricks marketing win, not open-source control
“I think probably the biggest thing that they get out of the acquisition is the ability to tell the market, That they can do iceberg two and not look like they had made a mistake with Dell with Delta. You know, that, that they can, they get sort of a marketing …”
Habib predicts AI will be able to run companies within five years
“This is AI that in three to five years is going to be able to run companies.”
Benaich: The AI existential risk debate was overblown
“I felt like the extra risk kind of debate was kind of overblown .”
Bernhardsson: GPU alt-clouds are real estate businesses adding little stack value
“It's like sort of like almost like real estate, right? Like they own sort of, you know, real estate and they rent it out, arguably sort of, you know, we work, you could say, if you want to be, you know, a little cynical, but like not a ton of value, like highe…”
Delangue: China is likely the current leader in open-source AI
“China has been publishing much more open source AI lately. I would argue that they're probably the leader today of open source AI which is surprising to some, but for example, in, in video, they've been leading in, in open source AI.”
Delangue: Traditional software startup rules are outdated for AI
“A lot of the things that we learn with traditional software are like outdated with AI.”
Socher: AI scaling will hit logarithmic leveling off in reasoning gains
“I do think we are going to hit a sort of logarithmic kind of leveling off off like how much more data will give us that much more reasoning, right?”
Socher: Nvidia will match AI hardware startups' 100x gains before startups reach scale
“By the time you get that out and it's actually scalable and it can really train it and all the software is ready for it. And I can now go on AWS and spawn up your new hardware, like, Nvidia will also have been a hundred X faster with their latest and greatest …”
Socher: Half of competitors' AI search citations are random, irrelevant links
“Some of our competitors, half of their citations are Independent, random links that have nothing to do with the sentence that they're behind.”
Pomel: Paul Graham's 'Founder Mode' essay will do more harm than good
“My worry about the founder mode, and everything that gets pulled down to a very short piece like that, is that I think it's going to be used and abused in all sorts of different ways, because there's so much context that goes into every single word in there, l…”
SDR and CSM roles often serve as a crutch, says Aaron Katz
“In my experience, it can often, both of those functions can be a crutch for a broader issue inside of the company.”
Scaling compute and GPUs alone won't yield autonomous enterprise AI
“I'm not an expert in the field, but I believe that this is gonna we need Another giant leap of innovation to get there. I'm not sure just throwing compute and increasing the number of GPUs without some fundamental discovery, it's gonna create this.”
Liu: Fully autonomous agentic AI approaches are fundamentally challenged
“And the key here is you can't automate end to end things with AI, right? And that's why I think like the agentic auto GPT like approach is really challenged because, you know, there's, it's just too hard for AI to kind of reason about every single step, break …”
Most major software companies will eventually integrate a Web3 database
“And so there's this asymptotic cost curve that will come down, and at some point, our belief is that most major software companies will actually have a Web three database as part of their stack.”
Evans: VR headsets will not replace smartphones as primary computing platforms
“You're not going to wear a headset, no matter how light and cheap it is, all day outdoors. I would not have worn it walking here, even if it weighed a hundred grams and had completely perfect pass-through. Therefore, it can't replace your phone. Therefore, we …”
Valenzuela: AI researchers face existential crisis as model scaling solves problems
“A lot of them are going through an existential crisis of sorts. It was maybe a lot of things you were trying to solve and work are just going to be solved by training larger models.”
AI content creation now surpasses the bottom 80% of marketing employees
“AI can generate content that has gone from laughably bad to, like, better than the bottom 25% of marketing employees to, like, maybe better than the bottom 80% of marketing employees and on a trajectory to continue progressing there.”
Traynor: There is no line of sight to a trillion-dollar European company
“Like I don't see a line of sight to a trillion dollar company in Europe full stop.”
dbt CEO Tristan Handy: The modern data stack concept is dead
“I don't think the modern data stack is a useful idea anymore.”
Eiso Kant: AI will recycle world data into synthetic data within years
“And so this is kind of our view and our, if you extend this a couple of years out, and this is where some people will probably have objection with us. We think that this will go so far. That we will kind of recycle all of the world's real data into higher qual…”
Srinivas: Google is making a mistake chasing OpenAI with Gemini
“Hence why, like, you know, I actually think Google is making a big mistake by trying to do whatever OpenAI is doing, right, like, going after them with the large, oh, I'm not, they're having GPT-IV, I'm going to turn Gemini.”
Srinivas: Building a web search index is harder than competing with GPT-4
“We believe that building your own index is even harder than trying to compete with GPT-IV because it's not a problem that's just solved with money.”
Srinivas: Citing web sources in AI search constitutes legal fair use
“And it's actually fair use. The fact that you're attributing credit is fair use.”
Mathew Lodge argues prompt engineering is a fallacy and merely trial-and-error
“And so the idea that there's something called prompt engineering, engineering in the sense of you're applying an approach because it's going to give you the output that you desire. Is a complete fallacy. At best it's trial and error. It's prompt trial and erro…”
Goshen: Scaling LLM size is already showing diminishing returns
“And we're already start seeing diminishing returns. I mean, you take the architecture, you increase the size of the models or, and you see that At these levels of scale where, you know, we are dealing with things, diminishing returns, we start seeing it.”
Shah: AI models attempting medical diagnoses will kill patients
“I mean, they just keep going there. I'm going to do diagnoses because it's grand or it's challenging, but I mean, they're going to kill somebody.”
Socher: You.com solved AI hallucinations before anyone else
“We figured that, I don't know, problem of hallucinations out before anyone else.”
Socher: Microsoft Bing copied You.com's AI image generation feature
“That was eventually copied, and Bing also has that feature now within Bing.com.”
Socher: Plumbers will eventually earn more than marketers as AI automates knowledge work
“Probably at some point, your plumber might get more than your marketer because the marketer can use tools and be way, way more efficient, but the plumber AI is going to take a very long time because None of the plumbing companies are collecting data, having ro…”
Riparbelli: LLMs are not production-ready for 95% of intended tasks
“They're not production ready for 95% of the tasks that people think that they want to use them for.”
Riparbelli: Most harmful AI misuse will emerge from open source models
“And I think we'll see that a lot of harmful use of these technologies will emerge from kind of like the open source world.”
Alex Ratner: Traditional model-centric AI development is dying off
“Tweaking the model, finding a fancier algorithm, all this kind of traditional model-centric development that, frankly, we still mostly teach in data science one-on-one courses as what data science is, is effectively dying off.”
Wiggins: Data science, not physics, won World War II
“I grew up as a physicist thinking that, you know, physics really won World War II, but now that I'm a data scientist, I realize that it was actually data science that won World War II, but that story was classified for about 75 years, which is the story of how…”
Liberty: Prompt engineering is a fad, not a long-term profession
“A lot of people believe that prompt engineering is incredibly crucial. I don't necessarily buy into that. I don't know if you remember, but people, like, thought that, like you know query building for Google would be a profession too, right? And that didn't qu…”
Falcon: Open-source AI will quickly catch up to proprietary models
“So I think all, all roads lead to open source at the end of the day. No matter what you do or how much money you have, you cannot compete with the world's resources put together to do something. So some companies will have an edge for a bit, yes, but open sour…”
Falcon: Scaling massive AI models compensates for gaps in scientific understanding
“I kind of think that the whole large scale model thing, It's really a gap in our scientific understanding of these models. Instead of looking at the math and coming up with a better loss function, we figured out that we could throw a compute at it, and get us …”
Falcon: Mathematical advances will eventually make future AI models small
“I think the math field will catch up, and we'll have some fancy regularizer that will make all this go away, and now we're back to, like, a small model. So I guess, I think that the future is small, large language models, I guess, that are specialized”