AI-driven targeting has turned battlefield judgments from categorical, accountable decisions into opaque probabilistic scores (like a 73% chance someone is a Hamas terrorist), which normalizes accepting false positives and erases individual accountability for lethal mistakes.
Carson describes how modern military AI (e.g., Palantir-style systems) replaced binary combatant/non-combatant judgments with opaque probability scores, meaning commanders accept a known false-positive rate without understanding how the number was derived or who is accountable for it. ✦ AI generated
Brad Carson · Machine Learning Street Talk · 2026-05-31 · original ↗
starts at this moment · 23:58
“What do you think the solution's going to be or how is this going to play out... it's happening right now in war, right?”
Now it's a gradient. You're on a heat map somewhere. You know where if you're in Gaza, Keith, you have a 73, you know, percent that you're a Hamas terrorist. And you know what makes that, you know, what is 73 like you get struck for that or are you off the list for that? Like what's the threshold?
verbatim transcript · starts at 23:58
23:40Um, we know that they have failure modes as you said that we don't really anticipate and can't identify. There's no person to hold accountable. And you know, I think there's a lot of worrying things. You know, I was talking to somebody the other day who I knew from the Pentagon. They said, you know, we came of age in a war in a world, and this has been true since the 1870s when
23:58the law of war really took off after the Crimean conflict where things were categorical. You're a legitimate target or you're not a legitimate target. You are a um, you know, a civilian or you're a combatant. These are things that people talked about as categorical binary almost. Now it's a gradient. You're on a heat map somewhere. You know where if you're in Gaza, Keith, you have a 73, you know, percent that you're a
24:24Hamas terrorist. And you know what makes that, you know, what is 73 like you get struck for that or are you off the list for that? Like what's the threshold? >> You know, it's no longer binary and categorical. it's on a gradient and um it's people don't understand what that even means. There's a lot of judgment goes into it. And so now we've created this world which is not true before
24:47where like we're accepting false positives as part of the game. You know, I'll prioritize false positives. You know, it used to be mistakes happened, but you didn't think you were [clears throat] making mistakes. You didn't say like, hey, there's an 80% chance that's Keith across the battlefield. I'm going to shoot him. It's like, no, Keith's a combatant. I can shoot him dogmatically, categorically, he's a combatant. I'll
25:05shoot him. That could be an error, but that's how you thought about it. Now it's like, well, there's some percentage that Keith is a combatant that my Palunteer, you know, interface is telling me and he is don't really understand how that number came to be, but it's 73 commander is 73 above the threshold or below the threshold. And we know in 27% of the cases it's going to
25:24be wrong, you know. So, what what's our false positive rate we're accepting here, commander? That's the kind of thing that's happening. And yeah, I don't love it. And in fact, I think if um you know, we need to get our act together and try to restrict it as much as possible. There's still going to be places where those deterministic systems could work. You know, missile defense, offensive cyber or defensive cyber
25:44rather, where you know, you have to respond in in sub-second kind of time frames. That's fine. But those are a far different kind of AI than the introduction of the neural net to this world. >> Yeah. And I and I and I take the point that that neural networks, at least most of them, are inherently probabilistic. But on the other hand, the the sort of prior binary categorization, I think
- ·AI systems replaced binary combatant/non-combatant calls
- ·Targets now get a probability, like 73% 'Hamas terrorist'
- ·Commanders accept a known false-positive rate blindly
- ·Unclear how the score is calculated or by whom
- ·No clear threshold for when a score triggers a strike
- ·Probabilistic scoring erases individual accountability
- ·False positives become normalized, routine outcomes