Video · 2026-05-31 · 1h 21m · 6 moments

The Ex-Pentagon Chief Sounding the Alarm on AI Weapons — Brad Carson

✦ AI generated

timeline · colored by role

Claim

Regulatory capture arguments against AI oversight are unfalsifiable, and an accountable public agency subject to potential capture is still preferable to today's status quo where informal, wealthy networks of AI industry insiders already control policy without any accountability.

Carson likens the anti-regulation 'capture' argument to searching for a pea under 100 mattresses — always assertable, never falsifiable — and argues that groups like a16z already informally capture AI policy, making a formal agency (like the SEC model) more accountable, not less.

transcript

Brad Carson: My view is that groups like A16Z or people in Silicon Valley have largely captured the process because they work very informally through networks of political influence... having an agency subject to regulatory capture yes is at least more accountable than the kind of informal very moneyed networks that are controlling AI policy as we sit here in 2026.

rebuts · 1

Claim◆

AI language models are machines, not persons, and giving them speech rights or treating their outputs as protected expression is dangerous because it exploits humans' anthropomorphic tendency toward fluent language.

Brad Carson argues that because humans instinctively anthropomorphize fluent language, people mistake chatbots for persons (citing 'AI psychosis'), which is dangerous when it leads to granting AI outputs First Amendment-style rights instead of treating them as regulable products.

transcript

Brad Carson: We know across the board the anthropomorphic tendency we have when we see language. We love language. We think it's the unique human skill. And when this machine turns it out, we know through AI psychosis and other things that people think it's a person. And therefore, they're giving the rights of persons to something. And that to me is a very dangerous thing. But it's a machine and we should treat it like a machine.

Claim◆

AI-driven targeting has turned battlefield judgments from categorical, accountable decisions into opaque probabilistic scores (like a 73% chance someone is a Hamas terrorist), which normalizes accepting false positives and erases individual accountability for lethal mistakes.

Carson describes how modern military AI (e.g., Palantir-style systems) replaced binary combatant/non-combatant judgments with opaque probability scores, meaning commanders accept a known false-positive rate without understanding how the number was derived or who is accountable for it.

transcript

Brad Carson: Now it's a gradient. You're on a heat map somewhere. You know where if you're in Gaza, Keith, you have a 73, you know, percent that you're a Hamas terrorist. And you know what makes that, you know, what is 73 like you get struck for that or are you off the list for that? Like what's the threshold?

Claim◆

The belief that an AI arms race is inevitable and unstoppable is false and dangerous; societies have repeatedly chosen to restrain powerful technologies (bioweapons, germline editing, cloning) and can do the same with AI, especially since the West controls the chip supply chain.

Carson rejects AI fatalism, pointing to historical precedents (nuclear arms treaties, banned bioweapons, halted germline editing) where humanity chose restraint, and argues chip-supply-chain leverage gives the US real power to shape AI's trajectory rather than just accept it.

transcript

Brad Carson: It may not be the thing we want to do, but it is something that's doable and we shouldn't just like resign ourselves to say, 'Hey, our fate is out of our hands.' We control it. And in fact, that might be the most dangerous view I hear at AI where people say, 'It's just coming like a freight train. Nothing you can do about it.' I don't accept that.

rebuts · 1

Claim

The American way of war mistakenly substitutes capital (technology) for labor (people), overestimating what advanced kit like AI can accomplish when only human beings can actually occupy territory and rebuild governance.

Drawing on Iraq, Afghanistan, and Iran, Carson argues the US repeatedly over-relies on technology as a substitute for manpower, when history shows only human beings can occupy territory and install new governments — a lesson he fears is being repeated with AI.

transcript

Brad Carson: How you win wars is with people, you know, that's the fundamental thing... The American way of war in many ways is substituting capital for labor. We love bright, shiny objects. We think they're technical solutions to vexing human problems. And we're always betrayed by that because in the end when you go to Iraq, you go to Afghanistan, you go to Iran, all the fancy kit, they can reduce your city to rubble, right? But the only thing they can't do and only humans can do this is basically come kick in your door and occupy your place.

Claim

Anthropic, as a private vendor, has every right to set and enforce its own terms of service with the Pentagon — including refusing certain uses — just as any vendor (like a lawn-care company) can decline a customer, since Claude is not a monopoly utility and competitors exist.

Addressing the Anthropic-Pentagon dispute, Carson argues Anthropic is acting as a normal commercial vendor setting contract terms, not as a monopoly utility, so it has the right to refuse or restrict how the Department of War uses Claude — the real fix is Congress clarifying what counts as 'lawful use.'

transcript

Brad Carson: If my lawn guy says, 'I don't want to be your lawn guy anymore, right? Because I don't like what you do or you believe or you're working in something. I just don't want to be your vendor anymore.' I shouldn't be able to compel him to be my lawn guy. I go find another lawn guy.

rebuts · 2

Highlight slides

Language Fools Us Into Seeing a Person✦ from: AI language models are machines, not persons, and giving them speech rights or treating their outputs as protected expression is dangerous because it exploits humans' anthropomorphic tendency toward fluent language.Why It's Dangerous✦ from: AI language models are machines, not persons, and giving them speech rights or treating their outputs as protected expression is dangerous because it exploits humans' anthropomorphic tendency toward fluent language.From Judgment to Score✦ from: AI-driven targeting has turned battlefield judgments from categorical, accountable decisions into opaque probabilistic scores (like a 73% chance someone is a Hamas terrorist), which normalizes accepting false positives and erases individual accountability for lethal mistakes.Who's Accountable for 73%?✦ from: AI-driven targeting has turned battlefield judgments from categorical, accountable decisions into opaque probabilistic scores (like a 73% chance someone is a Hamas terrorist), which normalizes accepting false positives and erases individual accountability for lethal mistakes.AI Fatalism Is a Choice, Not a Fact✦ from: The belief that an AI arms race is inevitable and unstoppable is false and dangerous; societies have repeatedly chosen to restrain powerful technologies (bioweapons, germline editing, cloning) and can do the same with AI, especially since the West controls the chip supply chain.Precedents for Restraining Powerful Tech✦ from: The belief that an AI arms race is inevitable and unstoppable is false and dangerous; societies have repeatedly chosen to restrain powerful technologies (bioweapons, germline editing, cloning) and can do the same with AI, especially since the West controls the chip supply chain.

Related episodes

Video · The Cognitive Revolution

Fable's Back, AI Engineer Recap, & SambaNova

→

Video · a16z Podcast

Ben Horowitz on the Global Race for Tech, Power, and Influence

→

Video · Lenny's Podcast

The rise of taste, human authenticity and judgment in an AI world | Adam Mosseri (Head of IG)

→