My core disagreement with Eliezer Yudkowsky is about moral realism: I believe a sufficiently intelligent system converges on cosmopolitanism, pluralism, and cooperation because that is genuinely the correct strategy, whereas Yudkowsky thinks human values are just one arbitrary point among millions of possible coherent extrapolated volitions.

89:05there's some function of the state of the world that it wants to maximize the expected value of and there's a lot of theory that you know the complete class thems and the Morgan string theorems and the Dutchbook theorems and all the rest that all suggests that any agent that isn't trying to maximize the expected value of some state of the world is going to get eaten. And so when I talked

89:28to Eleazar about this, which I haven't in many years, but when I did, he would say like, okay, so yeah, like maybe there will be like this coalition of weak sauce AIs, but like they're going to get eaten by the the the the you know, the actual strong AIs that are doing optimization. So yeah, I think there's there's some, you know, uh what what you could call a non-scientific

89:51question, but from the evolutionary game theory point of view, it kind of is a scientific question. And from the aausal view, it's kind of a mathematical question, which is like, is there a dominant strategy for how to do well in the universe or in the multiverse? And I think there is a dominant strategy and it involves cosmopolitanism and pluralism and cooperation and you know mutual [snorts] information and truth

90:19and harmony and all you know kind of all the all the good things that human culture has discovered flow from this this kind of this is the right strategy for how to be and thus a sufficiently intelligent system would figure it out. And all of the all of the drama comes in the kind of adolescence of developing some capabilities ahead of others. Whereas for Eleazar, all of the you know

90:45the the kind of alignment of what a system is trying to do is arbitrary. And humans have a particular kind of collection of values that for Eleazar I think I don't want to say this too strongly because I haven't gone you know haven't had this conversation but I think he would say the reason that human values are worth installing is that they are our values and so we do better according to them by