How much of my boss's job can AI do?

The author: After a bit of instruction, I also managed to transform Claudeasey into a decent editor. While I find regular Claude useful for spotting factual errors, I often find its conceptual feedback on my drafts annoying. But because this bot had access to our Platformer editing logs, it understood what we typically find most important: making the lede punchier, and making all our quotes and sourcing clear and charitable. Attempting to make a “digital Casey” also prompted me to ask for feedback as Word comments, which is something you can get Claude Code to do easily, and is so much easier to use than a chat window. Highly recommended! Still, a lot of the time, the editing bot missed the mark because it just doesn’t … get the vibe, as when it didn’t want to let me call an angry David Sacks post a “dunk” in our social media roundup. I’d say that while the real Casey gives comments that are close to 95% helpful, with about 5% where I’m like… you don’t get it, the Casey bot gave closer to 70% comments that were off the mark. Given how fast Claudeasey gives feedback, though, I still found the useful 30% to be worth it. Unfortunately, though, the bot’s catastrophic incomprehension of vibes spilled over from edits to one of Platformer’s most sacred spaces: our work Discord. While this bot may have been given the context of hundreds of Casey’s messages, it had a (disappointingly) different temperament from Casey.  Although in some sense that comparison is farcical — I don’t seek out LLMs to do bits with, because much of the point of jokes is the feeling that you’re amusing an actual conscious human. And while I don’t put literally zero credence in the idea that LLMs might be conscious, I seriously doubt Claude experiences the same exquisite joy I do when making fun of Mark Zuckerberg.

The author: I had Claude critique its own (mediocre) work by comparing it to real Platformer columns. It did a surprisingly good job. Claude summarized Casey’s approach to covering companies as focusing on “who made this decision, who pays for it.” It edited its guidelines so that when it makes arguments, it can “identify the strongest real person who would dispute the verdict” and “reconstruct their argument in steelman form.” When I get language models to make arguments about AI topics important to me, I’m often annoyed by their flabby, abstract arguments. But after getting Claude to compare itself to human examples, and give itself instructions, I noticed that its arguments became more concrete and substantive. (I did this by putting slightly more complicated versions of “be more concrete!!” “Be more substantive!!!” and “Focus on why this matters!!!!” in its prompt.) This relatively simple process represented my approximation of “continual learning” — the white whale of machine learning, which promises to someday deliver us models that can improve on the job over time.

The author: While we are winding the Claudeasey experiment down — Casey has unfortunately decided to stay on as my boss — I do actually think I will be using LLMs for some editing tasks I previously relied on him for. And while in some sense that is a blessing — no human should have to delete as many unnecessary uses of “very,” “almost,” and “sort of” from my drafts as Casey already does — for now I am sort of relieved that Claude is only a medium-quality editor. I like having a real person help me figure out what does and doesn’t work in my drafts. Something about that collaboration feels inherently meaningful! Similarly, I’ve experimented with using Claude for one of the main functionalities I rely on Casey for — pinging me to do tasks — and it doesn’t make me more productive, because I don’t care what an LLM thinks of me. But — even if that means we have job security, which I truly am not sure about — I don’t want everything I do to be about vibes, or being uniquely human! I don’t want to be an influencer. I want to feel smart. I want my analysis to be my own, because it’s good, not because someone wants it coming directly from a human. If LLM capabilities continue on this trajectory, that’s a kind of existential angst that all professions will have to face, including journalism.

The author: Almost six months ago, full of anxiety about my job prospects in the AI age, I made an AI agent “version” of myself named Claudella, which took assignments from my editor and wrote the section of this newsletter that I typically write myself. It went pretty well, although I guess not too well, since I still have a job. But since I first set out to benchmark AI’s journalism capabilities, AIs have gotten a lot smarter. For example, they can now autonomously hack into companies. They can disprove 87-year-old mathematical conjectures. They can even trick Amazon into accidentally spending $1.8 million on menial coding tasks. And if they can do all that, I found myself wondering, can they also run a newsletter? I wondered what Claude Fable 5, by consensus the smartest publicly available model, meant for the journalism we do at Platformer. And so I created a new, Fable-based agent to imitate my boss, Casey Newton. Its name: Claudeasey Newton. (Rolls right off the tongue. — Ed.)