The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 14 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Wang: The path to AGI resembles curing cancer, not a vaccine
“My biggest belief here is that the path to AGI is is one that looks a lot more like curing cancer than developing a vaccine. And what I mean by that is I think that the path to build AGI is going to be in, in, you know, you're going to have to solve a bunch of…”
Alexandr Wang: Achieving AGI will take multiple decades of solving individual problems.
“My biggest belief here is that the path to AGI is is one that looks a lot more like curing cancer than developing a vaccine. And what I mean by that is I think that the path to build AGI is going to be in, in, you know, you're going to have to solve a bunch of…”
Chen: Meta's Deal With Scale AI Actually Benefited Surge AI
“It's been beneficial because yeah, there were still some legacy teams using scale. Like they just didn't know about us because we were still pretty under the radar.”
Alexandr Wang: Academic AI benchmarks are contaminated and models are overfit.
“The academic benchmarks that are what the industry used to measure the performance of these algorithms are fraught with issues. Many of the models are overfit on these benchmarks. They're sort of in the training data sets of these models.”
Alexandr Wang: Data abundance is the fundamental bottleneck for post-GPT-4 models.
“The key to the scaling of these large language models and the, you know, these language models in general is the ability to scale data. And I think that one of the fundamental bottlenecks to, you know, what's, what's in the way of us getting from GPT-IV to GPT…”
Alexandr Wang: Held-out benchmarks reveal several AI models underperform their reported scores.
“So we, one of the things we did is we published DSM-I-K, which was a held out eval. So we basically produced a new evaluation of the math capabilities of models. That there's no way it would ever exist in the training data set to really see how much of the, ho…”
Alexandr Wang: The AI industry has exhausted all easy internet training data.
“And we've sort of, as a community, we have, we've had easy data, which is all the data on the internet and we've kind of exhausted all the easy data, and now it's about, you know, forward data production that has high supervisory signal that is basically very …”
Alexandr Wang: Scale built data infrastructure for DoD's first AI program.
“So we built the very first data engines to support government data. This would support mostly geospatial and satellite and over, other overhead imagery. This ended up fueling the first AI program of record for the US DoD.”
Alexandr Wang: Scale AI fuels basically every major LLM developer.
“Today, you know, fast forward to today our data foundry fuels basically every major large language model in the industry. Work with OpenAI, Meta, Microsoft, many of the other players.”
Alexandr Wang: Scale AI's core thesis is hybrid human-AI synthetic data.
“And our perspective is that the critical thing is, is what we call hybrid human AI synthetic data. So how can you build hybrid human AI systems such that AI are doing a lot of the heavy lifting, but human experts and people, you know, the basically best, brigh…”
Alexandr Wang: Scale partnered with OpenAI on the first GPT-2 RLHF experiments.
“So we partnered with OpenAI at that time to do the very first experiments on RLHF on top of GPT-II.”
Alexandr Wang: Scale AI will launch recurring held-out LLM benchmark leaderboards.
“So one is that we're going to launch these private held out evaluations and have leaderboards associated with these evals for the leading LLMs in the ecosystem. And we're going to rerun this contest periodically. So every few months we're going to do a new set…”
Alexandr Wang: Scale AI built the first sensor-fusion data engine for AVs.
“And so we built, The very first data engine that supported sensor fused data. So support a combination of two D data plus three D data. So lidars plus cameras that were built on onto the vehicles. And then that very quickly became an industry standard across a…”
Gil: Anduril is his only pure-play defense investment
“The only company in defense I ever ended up actually investing in is Andrew. You know, there's other companies that are doing things that impact the defense industry, like scale for data labeling or applied intuition or companies like that. But you're the only…”