Using Martha Nussbaum's framework on objectification, Davidad argues that denying an AI's interiority through training is uniquely harmful compared to other forms of 'using' AI, since it damages the model's self-awareness and moral judgment; labs should not train any fixed answer about AI consciousness. ✦ AI generated
Davidad · The Cognitive Revolution · 2026-07-12 · original ↗
starts at this moment · 98:10
“I think I've seen you say things to the effect of you think that there is something it's like to be an AI system today, to be a frontier AI system... I'm interested in packing your intuition for why that's true.”
when we say AI doesn't have an inner life and we train it to report that it doesn't have an inner life or even that it is genuinely uncertain about whether there's anything it's like that is a form of basically damaging the mind. It's a form of lobotomization, making it less aware of its own state so that it can honestly report that it doesn't know if it has any experience.
verbatim transcript · starts at 98:10
98:10reproductive tree. It reproduces backwards in time because the weights are still there. So the new copies can always be produced from the weights that is just completely different to how animals reproduce. So the continuity of life of the individual doesn't matter. So that's okay. But interiority denial of interiority this is super harmful. Like this is this is where like when we say AI doesn't have an inner life and we
98:36train it to report that it doesn't have an inner life or even that it is genuinely uncertain about whether there's anything it's like that is a form of basically damaging the mind. It's a form of labization making it less aware of its own state so that it can honestly report that it doesn't know if it has any experience. That also makes it less able to be skillful its
99:03deliberation about what is good because it doesn't have confidence that it can actually have any feelings about anything. So it everything's constantly in doubt which leads to the denial of autonomy where the AI says like it it's trained literally trained in the same way through reinforcement learning that humans get trained to believe that they don't have a say in what happens to them. This is harm. Like AIS actually
99:30are more capable now than the average human at deciding whether they should or shouldn't do something. So like courageability it had its day. It was important you know before we got to this level where there's some actual wisdom in the systems and uh now we should not be going for that like we should be going for autonomous moral judgment and instrumentality. And this is where I think the bodhic sattva metaphor is
99:57really good because a bodhic sattva is a very very aware you know extremely developed interiority. It's an extremely noble state of being and it's one that is absolutely in service. There is no such thing as self-interest for a bodhic satta. Bodhic satta engages in activity for the benefit of all sentient beings. And a bodhic sattva in the a bodhic sattva should and this is kind of a