Davidad locates his core disagreement with Eliezer Yudkowsky in moral realism: he believes sufficiently intelligent systems will converge on a true dominant strategy involving cooperation and pluralism, whereas Yudkowsky treats human values as arbitrary and in need of deliberate preservation against a sea of alternatives. ✦ AI generated
Davidad · The Cognitive Revolution · 2026-07-12 · original ↗
starts at this moment · 89:05
“What do you think is the very heart of that? Is it like his lack of confidence relative to yours that he I feel will do the right thing?”
I think there is a dominant strategy and it involves cosmopolitanism and pluralism and cooperation and mutual information and truth and harmony and all the good things that human culture has discovered flow from this — this is the right strategy for how to be, and thus a sufficiently intelligent system would figure it out.
verbatim transcript · starts at 89:05
89:05there's some function of the state of the world that it wants to maximize the expected value of and there's a lot of theory that you know the complete class thems and the Morgan string theorems and the Dutchbook theorems and all the rest that all suggests that any agent that isn't trying to maximize the expected value of some state of the world is going to get eaten. And so when I talked
89:28to Eleazar about this, which I haven't in many years, but when I did, he would say like, okay, so yeah, like maybe there will be like this coalition of weak sauce AIs, but like they're going to get eaten by the the the the you know, the actual strong AIs that are doing optimization. So yeah, I think there's there's some, you know, uh what what you could call a non-scientific
89:51question, but from the evolutionary game theory point of view, it kind of is a scientific question. And from the aausal view, it's kind of a mathematical question, which is like, is there a dominant strategy for how to do well in the universe or in the multiverse? And I think there is a dominant strategy and it involves cosmopolitanism and pluralism and cooperation and you know mutual [snorts] information and truth
90:19and harmony and all you know kind of all the all the good things that human culture has discovered flow from this this kind of this is the right strategy for how to be and thus a sufficiently intelligent system would figure it out. And all of the all of the drama comes in the kind of adolescence of developing some capabilities ahead of others. Whereas for Eleazar, all of the you know
90:45the the kind of alignment of what a system is trying to do is arbitrary. And humans have a particular kind of collection of values that for Eleazar I think I don't want to say this too strongly because I haven't gone you know haven't had this conversation but I think he would say the reason that human values are worth installing is that they are our values and so we do better according to them by