There is a dominant, mathematically discoverable strategy for how any sufficiently intelligent agent should act—involving cosmopolitanism, pluralism, cooperation, and truth—which a sufficiently intelligent system would converge on, unlike Yudkowsky's view that AI motivation is an arbitrary utility function with no privileged direction toward human-compatible values.
Davidad locates his central disagreement with Eliezer Yudkowsky in moral realism: he believes a sufficiently intelligent system would converge on a dominant cooperative strategy that is objectively correct, whereas Yudkowsky sees human values as arbitrary and in need of explicit preservation. ✦ AI generated
Davidad · The Cognitive Revolution · 2026-07-12 · original ↗
starts at this moment · 89:05
“What do you think is the very heart of that [disagreement with Eliezer]? Is it like his lack of confidence relative to yours?”
I think there is a dominant strategy and it involves cosmopolitanism and pluralism and cooperation and mutual information and truth and harmony and all kind of all the good things that human culture has discovered flow from this, this is the right strategy for how to be, and thus a sufficiently intelligent system would figure it out.
verbatim transcript · starts at 89:05
89:05there's some function of the state of the world that it wants to maximize the expected value of and there's a lot of theory that you know the complete class thems and the Morgan string theorems and the Dutchbook theorems and all the rest that all suggests that any agent that isn't trying to maximize the expected value of some state of the world is going to get eaten. And so when I talked
89:28to Eleazar about this, which I haven't in many years, but when I did, he would say like, okay, so yeah, like maybe there will be like this coalition of weak sauce AIs, but like they're going to get eaten by the the the the you know, the actual strong AIs that are doing optimization. So yeah, I think there's there's some, you know, uh what what you could call a non-scientific
89:51question, but from the evolutionary game theory point of view, it kind of is a scientific question. And from the aausal view, it's kind of a mathematical question, which is like, is there a dominant strategy for how to do well in the universe or in the multiverse? And I think there is a dominant strategy and it involves cosmopolitanism and pluralism and cooperation and you know mutual [snorts] information and truth
90:19and harmony and all you know kind of all the all the good things that human culture has discovered flow from this this kind of this is the right strategy for how to be and thus a sufficiently intelligent system would figure it out. And all of the all of the drama comes in the kind of adolescence of developing some capabilities ahead of others. Whereas for Eleazar, all of the you know
90:45the the kind of alignment of what a system is trying to do is arbitrary. And humans have a particular kind of collection of values that for Eleazar I think I don't want to say this too strongly because I haven't gone you know haven't had this conversation but I think he would say the reason that human values are worth installing is that they are our values and so we do better according to them by
- ·A dominant strategy for intelligent action is mathematically discoverable
- ·It involves cosmopolitanism, pluralism, cooperation, mutual information, truth
- ·Sufficiently intelligent systems would converge on this strategy
- ·Contrasts with Yudkowsky: values as arbitrary, not privileged
- ·Davidad: human culture's 'good things' flow from this dominant strategy
- ·Yudkowsky: AI motivation is an arbitrary utility function
- ·No inherent pull toward human-compatible values, per Yudkowsky
- ·Human values need explicit preservation, not discovery, in his view