Emergent, misaligned centers of economic growth don't need to answer to human desires at all, any more than a growing human city needs to answer to monkeys who think they're trading partners with it.

26:23government that we build is going to be like ultimately sort of being called shots will be being called by a human. And I always it just makes me think of like some monkeys and they're like trading bananas amongst each other and they see humans start to like build their city and they're like oh wow like you know we could probably trade with those humans and get rich.

26:40>> But of course ultimately what matters is the banana monkey economy and it might be hard to measure GDP if we don't count the human activity. But uh ultimately it's going to be the monkeys that are calling the shots because they're going to need to trade with us for bananas. And it's just like uh people just don't understand that they might be irrelevant someday as like consumers or producers.

26:57And there can just be complete it's just not that hard for them to be self-contained sources of growth that just don't have to answer to any particular human desire. And I mean governments is the classic one, right? Like the North Koreans never were like, "Hey, let's all get together and make a horrible system of government that oppressed us." And I don't think that like the Kims either

27:15were like, "Oh yeah, let's uh make this like horrible equilibrium. It just kind of happened." Same with like the USSR. Same with like all kinds of states throughout history. Um it's just so easy for you to like to accidentally build a layer of agency on top of you that doesn't actually care about you. It cares about growth and power just emergently. [music] The optimistic version goes, even if we

27:35hand the machines the wheel, broadly aligned AI keeps us comfortable. The machines of loving grace future. We put that to David. His worry isn't that we'll feel useless. >> The thing that I'm worried about is starvation. Like whether like us all feeling like we don't have meaningful jobs or whatever. Like that's not a serious problem in my point of view compared to literally not being able to

27:56eat enough or maybe like being forced to be uploaded so that you have a much smaller footprint, but then also on like very unfavorable terms so that you don't get to choose when you run and maybe you just only get trotted out for special occasions or or never. Who knows? I kind of think that's what like a like [clears throat] sort of like the slowest possible observation looks like.

28:12So again, and so the reason people aren't happy is because they can see that there is a agency that's maybe like government or some other like giant thing that doesn't particularly care about their welfare. They cares much more about some other types of beings welfare and there's nothing they can do about this. And of course, if humans are still making decisions, then maybe there's some things they can do. But I