An AI enforcer that perfectly enforces society's officially stated rules (rather than the actual, imperfectly-enforced status quo) would be necessarily misaligned from what actually works for humanity — so the 'aligned' AI the labs are trying to build would effectively be the paperclipper, while the AI that's actually good for humanity would have to be the misaligned one.
Prakash argues that because everyday life diverges heavily from society's official rules (e.g., who actually goes to prison, how much corruption is tolerated), an AI that perfectly enforced stated values as written would be catastrophic — making the supposedly 'aligned' AI the real danger. ✦ AI generated
Prakash · The Cognitive Revolution · 2026-07-07 · original ↗
starts at this moment · 29:51
if you wanted an AI that can manage day-to-day reality, that AI is necessarily misaligned from the from the documents that you say you want it to be aligned to because necessarily like our day-to-day is not aligned with what we want. And so you have this thing where the AI that may work out for humanity will be the misaligned one. And the AI that supposedly the lab leaders are trying to create the aligned AI would actually be the paper clipper.
verbatim transcript · starts at 29:51
29:51day-to-day reality, that AI is necessarily misaligned from the from the documents that you say you want it to be aligned to because necessarily like our day-to-day is not aligned with what we want. And so you have this thing where the AI that may work out for humanity will be the misaligned one. And the AI that supposedly the lab leaders are trying to create the aligned AI would actually be
30:21the paper clipper because that aligned AI would then like look at these rules and say like well this is what you said you wanted to aspire to and so we're going to we're going to execute on these right and uh and and that that is the thing I think maybe I feel there's a sense of naive in in the lab and the lab leadership because and again they don't
30:43want to say it. Uh I wish I wish it' just come out and say it, right? I wish it' come out and say like, "Okay, look, if we have AI as a enforcer, some of these people are going to go to prison." And then that becomes like concrete for people. Uh but they don't want to say that because it's very it's very in your face and like they're like, "Oh, you
30:59know, democracy will still work out. You can still make democratic decisions, but what actually are you saying there?" Right? So that that that is what has always struck me. And I think I I think Run has been approaching the issue more and more trying to push it in there. Um I I I do feel the lab leaders always just beat around the bush on this. So that's that's one of the annoying parts
31:23of the of this conversation that they they they don't want to like come out and just say it outright, right? Yeah. I mean I think we might excuse me. We might need some sort of like uh mass pardon or you know if there's a president that uh would be just the right president to mass pardon everybody before the AI enforcement regime gets underway. We might have just the guy in office for
- ·Everyday life diverges heavily from society's official rules
- ·An AI enforcing stated rules exactly would be catastrophic
- ·Day-to-day reality is not aligned with our stated values
- ·AI matching official docs = the real 'paperclipper'
- ·AI that actually works for humanity must be 'misaligned'
- ·Labs' 'aligned AI' goal inverts what's actually safe