Pangram Labs' AI-text detector can confidently score a piece of writing as 0% human even when the author spent over 50 minutes making extensive, substantive edits throughout the entire piece, showing its binary verdicts shouldn't be treated as proof of purely-AI, uncritical authorship.
Nathan shows a Google Docs edit history where he substantially rewrote nearly every section of an AI-drafted intro over an hour, yet Pangram Labs still scored it 0% human, arguing the tool is accurate on average but shouldn't be trusted as definitive proof in individual cases. ✦ AI generated
Nathan · The Cognitive Revolution · 2026-07-07 · original ↗
starts at this moment · 124:44
it gave me still a zero and that I think is enough to say okay so if there were four there were four things two of them admitted two of them kind of contested one I'd say fair enough the other one I would say, No, a zero score is is wrong. Like, you you definitely should give me more than a zero.
verbatim transcript · starts at 124:44
124:44this document making changes and it gave me still a zero and that I think is enough to say okay so if there were four there were four things two of them admitted two of them kind of contested one I'd say fair enough the other one I would say, "No, a zero score is is wrong." Like, you you definitely should give me more than a zero. I'd say
125:11if you if you called that one 25% AI, even maybe 75% AI, I would call that kind of acceptable. 75% would seem high given the fact that I basically did rewrite almost every section of the thing. But I did keep the structure. I did keep the general, but of course, the structure was derived from my examples, too. So it's not like there was it's not like I didn't have any hand in that. Um
125:36but yes, I think you know what can we say? Overall Pangram is quite accurate. Um, and yet we have at least one example out of 400 or so essays where I think the zero score I would confidently assert is wrong and unfair and should not be the basis for like a pylon. you know, it would the the the crowd uh the digital mob would be like in the wrong for
126:06piling on somebody uh for passing off my um snowflake intro essay uh or you know for attacking it as being a AI slop output. I think I can show this edit history and everybody should agree that like yeah, you put in an hour, you basically rewrote almost every section and somehow Pangram still gave you a zero from this. I would say you should not you cannot convict in a you know
126:34reasonable doubt system purely based on this sort of thing and yet at the same time you can pretty much trust the panagram signal as a consu so as a consumer I think you can trust it as a judge I think you should be more cautious >> I wonder to what extent uh people reading AI slop get trained on AI slop and start writing like AI slop and so
- ·Nathan spent 50+ minutes editing an AI-drafted intro
- ·He substantially rewrote nearly every section
- ·Pangram Labs still scored the piece 0% human
- ·Tool called accurate on average, not per-case proof
- ·Nathan contested the zero score as wrong
- ·Binary verdicts can misjudge substantive human editing