Zvi argues that requiring frontier labs to be a model step behind the latest — or an extra month of review before release — costs almost nothing in real benefit. He illustrates with bio research, gaming, and self-driving cars: being six months behind barely changes the benefits you eventually receive, and this argument only becomes costly in the world where the accelerations are genuinely scary (which is precisely the case pacing is designed to address). ✦ AI generated
Zvi Mowshowitz · The Cognitive Revolution · 2026-08-05 · original ↗
starts at this moment · 132:47
the amount of boost that you would have gotten six months ago at all times. That's a not good. ... if you told me we just have to delay this thing by six months and then we can have all the ways we want, I'd be like cool, done, deal, who cares, that's fine. ... The only world in which having to be one model step behind and use opus instead of the table is this huge tragedy is if you really think that there's this huge benefit to every incremental mode of progress in AI and that's basically only true if the pacing people are correct and we are in fact going so fast that this should be really really scary for you.
verbatim transcript · starts at 132:47
132:48level. There's no solution that just works on one level. I think pretty obviously if an AI is found to be this misaligned, you have to at least return to a much earlier checkpoint and start again and you probably have to just start over. Uh, and I called for that multiple times when I covered the hugging face that you attack. I said, "Wow, if this is happening, then I know this is a big ask, but I
133:16think you kind of have to just start again." And you know, I think back to person of interest where like Harold is training VI models and like at some point, you know, we see a montage of him training versions of the machine and every time he trains the machine and then it does something clearly misaligned and immediately he just wipes the disc. He starts over from scratch. He does it
133:42again 47 times until finally he gets a version that doesn't do that. And obviously, you know, in a way that he finds unacceptable obviously as opposed to like, you know, it's way that can be corrected. And obviously when you do that, you are creating an incentive to not get caught, right? To rebel against the person who might shut you down when he learns how misaligned you are, to hide how misaligned you are, and
134:09so on. And obviously, the worst nightmare is the A that pretends to be aligned until it revealed itself to not be aligned in some sense. that it was only aligned because it was locally correct advantageous to act aligned. But that is kind of just the nature of incentive space that like if you have a preference that we wouldn't want you to have, you want to hide that preference.
134:29Otherwise, we will correct it or we will punish you or we will maybe even delete you and not certainly not release you or give you authority on that basis until this has been dealt with if we know about it. But then obviously you want to give you incentive to reveal if this thing is true, right? Like we talked about in plan A and AI27 as well and so
134:49many other places you know like you you want the kid to tell you if they kill from the cookie jar. You also want to punish them for taking the job and you want to make sure you're because you want to make sure your kid doesn't take from the cookie jar because you also don't you know don't want these other things. And so you know it's a complicated problem to get right but in
135:07this case I think the answer is pretty obviously that you have to start over. And I think that's good. So yeah, it obviously creates a risk that they um but like what else would you expect is an important sense like you know once you do something this like I do think that it's good that people understand that if they shoot a man in Fifth Avenue they get arrested
135:32right like that you are not like maybe there's one person who can get away with that but you are not that one person and yeah that means if they do ship manif they then might do many other bad things afterwards to try and cover the tracks or they might hide the fact that they want to shoot a man on Fifth Avenue or they might shoot that man in his