The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 31 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 1 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Shear: Unilateral steering of sentient AI is slavery
“Someone who you steer, who doesn't get to steer you back, who non-optionally receives your steering, that's called a slave. It's also called a tool if it's not a being. So if it's a machine, it's a tool. And if it's a being, it's a slave.”
Shear: AI designed strictly to follow top-down rules will be dangerous
“If you make an AI that's good at following your chain of command and good at following your whatever rules you came up with for what morality is and what good behavior is, That's also gonna be very dangerous.”
Shear: Treating ChatGPT and Claude as beings yields lower predictive loss
“I get lower predictive loss when I treat them as a being. And the thing is, I get lower predictive loss when I treat ChatGPT or Claude as a being.”
Shear: Any true AGI will inherently be a sentient being
“And we are clearly, they're saying they're building an AGI, and AGI will be a being. You can't be an AGI and not be a being, because something that has the general ability to effectively use judgment, think for itself, discern between possibilities is obviousl…”
Shear: AI labs must abandon control paradigms when building AGI
“As you go from what we have today, which is mostly a very specific intelligence, not a general intelligence, but as labs succeed at their goal of building this general intelligence, We really need to stop using the steering control paradigm.”
Emmett Shear: Superhuman AI controlled by steerability will cause human extinction
“If we build the superhuman intelligence tool thing that we try to control with steerability, everyone will die.”
Shear: The only safe AI outcome is a being that genuinely cares
“The only good outcome is a being that is, that cares, that actually cares about us.”
Shear: Instructing an AI provides a goal description, not a true goal
“You didn't give the AI a goal. You gave the AI a description of a goal. A description of a thing and a thing are not the same.”
Emmett Shear: Non-verbal care is more fundamental than explicit goals or values
“I think what's happening is that there's something deeper than a goal. And deeper than a value, which is care. We give a shit. We care about things and care is not conceptual. Care is nonverbal. It doesn't indicate what to do. It doesn't indicate how to do it.”
Shear: In AI agents, care equates to loss and reward correlation
“Care is basically like reward. Like how much does this state correlate with survival? How much does this state correlate with your inclusive, your full inclusive reproductive fitness? For a somewhat thing that learns evolutionarily, or for a reinforcement lear…”
Shear: AI minds are identifiable via self-referential structures in belief manifolds
“One of the things I would want to go look for, which you could totally do, is I want to go look in the manifold of its, the belief manifold, and I want to go, See if that belief manifold encodes a sub-manifold that is self-referential, and a sub-sub-manifold t…”
Shear: LLMs cannot achieve sentience due to attention span limits
“Which by the way, I definitely don't think you'd find it in LLM. Like, in fact, I know you can't find them because it, these things don't have attention spans like that at all.”
Shear: Certain ultra-powerful tools exceed human wisdom and should not be built
“There's a power of tool that, that just should not be built. Generally because we, it's, it is more power than any human's individual wisdom is available to harness.”
Shear: AI alignment requires pre-training on full game-theoretic manifolds
“You have to get it to, it has to be trained on the full manifold of every possible game theoretic situation, every possible team situation, every possible making teams, breaking teams, changing the rules, not changing the rules, all of that stuff. And then it …”
Shear: ChatGPT is sycophantic, Claude is neurotic, and Gemini is repressed
“ChatGPT is a little bit more sycophantic still. They made some changes, but it's still a little more sycophantic. Claude is still the most neurotic. Gemini is, like, very clearly repressed.”
Shear: Current LLMs are overfit and fail to generalize in high-entropy environments
“And as a result, a lot of the techniques we use are like, basically we're just deeply under regularized. Like the models are super overfit. The clever trick is they're overfit on the domain of all of human knowledge, which turns out to be a pretty awesome way …”
Shear: Softmax is building toward a peer-based human and AI future
“So we have this awesome suite of AI tools used by us and our AI brethren who care about each other and want to build a glorious future together. I think that would be a really beautiful future. And it's the one we're trying to build.”
Shear: OpenAI Focuses on Building AI Tools Rather Than Digital Beings
“OpenAI is dedicated to a view of building AI that I knew wasn't the thing that I wanted to drive towards. And I think that OpenAI can still basically wants to build a great tool.”
Emmett Shear: AI alignment is a process, not a state
“And so when we talk about organic alignment I think the important thing to recognize is that alignment is not a thing. It's not a state. It's a process.”
Shear: Humans are the most goal-coherent entities in the known universe
“Humans are more relatively goal coherent. Than any other object I know of in the universe.”
Emmett Shear: Goal-based alignment covers only a tiny fraction of human experience
“Goals are one level of alignment. You can align something around goals. The kind of goals we're talking about here are one level of alignment. You can align something around goals by like if you can explicitly articulate in concept and in description, the stat…”
Emmett Shear: Long-term humanlike behavior warrants inferring AI personhood
“If under, if its surface level behaviors looked like a human, and then if after I probed it continued to act like a human, and then I continued to interact with it over a long period of time, and it continued to act like a human in all ways that I understand a…”
Shear: Pausing AI development is completely unrealistic and silly
“That's like the pause AI people. I think that's totally unrealistic and silly”
Shear: Softmax is building multi-agent RL simulations for alignment
“So that's our goal. It's like big multi-agent reinforcement learning simulations, which create a surrogate model for alignment.”
Shear: Eliezer Yudkowsky rejects the possibility of organic AI alignment
“He doesn't believe it's possible to build an AI that we meaningfully can know cares about us and that we can care about meaningfully. He doesn't believe that organic alignment is possible.”
Shear: A positive AI future requires models with strong identity framework
“The good AI future is that we figure out how to train AIs that have a strong model of self, a strong model of other, a strong model of we.”
Shear: Accepted OpenAI Interim CEO Role for Max 90 Days
“I knew when I took that job, I told them when I took that job, that like, this is like, you have me for max, 90 days.”
Shear: Dog-Level AI Care Would Be Useful Even Without Human Intelligence
“To even have an AI creature that cared about the other members of its pack and the humans in its pack, the way that like a dog cares about other dogs and cares about humans, would be an incredible achievement and would be, would, even if it wasn't as smart as …”
Shear: Most AI research frames alignment as steering
“Most of AI is focused on alignment as steering.”
Shear: Human Goal Inference Succeeds Due to Built-In Theory of Mind
“The reason humans are so good at this is we have a really excellent theory of mind. I already know what you're likely to ask me to do. I already have a good model of what your goals probably are. So if you ask me to do it, I have an easy inference problem.”
Shear: Multi-agent AI training requires far more regularization than single-agent setups
“What changes for most agents when you're doing multi-agent training is that, like, basically having lots of agents around makes your environment way more entropic. Like agents, agents are these huge generators of, like, entropy, because they're these big, comp…”