Article · 2026-07-27 · 6 moments

🎙️ How I AI: Claude Opus 5 Review + Browser use in Codex + How Cursor and a Raspberry Pi makes AI fun

Your weekly listens from How I AI, part of the Lenny’s Podcast Network ✦ AI generated

Example◆

AI testing can be far more exhaustive than human testing because AI does not default to following the happy path, as demonstrated when Codex immediately found a blocking bug in an onboarding flow that had survived for months.

Claire describes how Codex's automated browser testing uncovered a blocking bug in her onboarding flow that had survived for months — because as a human tester she always unconsciously followed the happy path.

transcript

Claire: When Claire tests her own onboarding flow, she naturally follows the happy path. She fills out every required field, clicks 'next,' and never intentionally tries to break anything. Codex tested the flow as both a team and an individual, pushed on required-field edge cases, and immediately uncovered a blocking bug that had survived for months simply because Claire always completed the form correctly.

gives example · 2

Claim◆

Claude Opus 5 produces excellent work but is one of the most frustrating models to interact with directly because it is timid, apologetic, and unusually dependent on human approval — refusing straightforward actions like resolving a one-line merge conflict because the code belonged to 'someone else's branch.'

Claire describes how Opus 5 regularly defers decisions it could make itself, asks subagents to flag tasks for human review, and required her to repeat 'just do it' constantly — though it still finished first in her seven-model blind benchmark.

transcript

Claire: Opus 5 stands out less for the quality of its work than for the way it behaves. Claire found it timid, apologetic, and unusually dependent on human approval. During real coding sessions, it refused to resolve a one-line merge conflict because the code belonged to 'someone else's branch.' It also asked subagents to flag tasks for human review and regularly deferred decisions it could have made itself. Claire ended up repeating 'just do it' constantly.

supports · 1

Mechanism

Frontier models perform better and produce broader coverage when given room to decide how to approach a task rather than being handed a detailed checklist of steps.

Claire found that switching from listing 25 specific testing steps to simply saying 'QA the onboarding flow' produced broader coverage with fewer blind spots introduced by her own assumptions.

transcript

Claire: Frontier models often perform better when they are given room to think. When Claire first started using browser use, she would give the model a list of 25 things to test. Now she simply says, 'QA the onboarding flow,' and lets it decide how to approach the task. The result is often broader coverage, with fewer blind spots introduced by her own assumptions about what matters.

gives example · 2supports · 1

Example◆

Persona testing with browser use — having AI use a product as a specific persona like a PM coming out of a meeting — can reveal structural friction that synthetic user research misses.

Claire's husband EJ proposed having AI evaluate ChatPRD by role-playing specific users — a PM fresh from a meeting, an engineer picking up a PRD — which exposed a cross-thread reference flow problem Claire knew existed but had never experienced so acutely from the user's perspective.

transcript

Claire: Claire's husband, EJ, came up with the idea. Rather than asking AI to evaluate a product in the abstract, he suggested having it use the product as a specific person: a PM coming out of a meeting, an engineer picking up a PRD, or a team lead checking usage. In ChatPRD, this approach exposed a structural problem in the cross-thread reference flow. Claire already knew the issue existed, but she had never experienced it so clearly from the user's perspective.

gives example · 1

Prediction

The AI industry may be entering an intelligence overhang where raw capability improvements become table stakes and the conversation shifts toward speed, cost, infrastructure, open source, and specific kinds of intelligence.

Claire observes that new models arrive every week and benchmark scores keep rising, but most builders can no longer take advantage of every incremental improvement, signaling a shift in what differentiates models.

transcript

Claire: The AI industry may be entering an intelligence overhang. New models arrive every week, benchmark scores keep rising, and most builders can no longer take advantage of every incremental improvement. Claire expects the conversation to shift toward speed, cost, infrastructure, open source, and specific kinds of intelligence. Raw capability is starting to look more like table stakes than a meaningful differentiator.

gives example · 2supports · 1

Anecdote

You do not need to understand code deeply to build physical hardware projects with AI — a basic literacy level, like knowing just enough Spanish to get by in San Diego, is sufficient to guide and validate AI-generated work.

Maddie Reese compares her coding ability to tourism-level Spanish: she can read enough to spot incorrect wire specifications and tell whether the AI is heading in the right direction without writing functions from scratch, which was still enough to ship three working hardware projects.

transcript

Maddie Reese: Maddie compares her coding ability to knowing just enough Spanish to get by in San Diego. She can read parts of the code, spot an incorrect wire specification, and tell whether the AI is heading in the right direction. She is not writing functions from scratch. That level of literacy was still enough to ship three working hardware projects.

rebuts · 1

Highlight slides

AI vs. Human Testing: The Happy Path Problem✦ from: AI testing can be far more exhaustive than human testing because AI does not default to following the happy path, as demonstrated when Codex immediately found a blocking bug in an onboarding flow that had survived for months.Persona Testing with Browser Use✦ from: Persona testing with browser use — having AI use a product as a specific persona like a PM coming out of a meeting — can reveal structural friction that synthetic user research misses.Opus 5: Best Results, Worst Personality✦ from: Claude Opus 5 produces excellent work but is one of the most frustrating models to interact with directly because it is timid, apologetic, and unusually dependent on human approval — refusing straightforward actions like resolving a one-line merge conflict because the code belonged to 'someone else's branch.'The 'Just Do It' Tax✦ from: Claude Opus 5 produces excellent work but is one of the most frustrating models to interact with directly because it is timid, apologetic, and unusually dependent on human approval — refusing straightforward actions like resolving a one-line merge conflict because the code belonged to 'someone else's branch.'The ChatPRD Case Study✦ from: Persona testing with browser use — having AI use a product as a specific persona like a PM coming out of a meeting — can reveal structural friction that synthetic user research misses.Codex Found a Blocking Bug in Minutes✦ from: AI testing can be far more exhaustive than human testing because AI does not default to following the happy path, as demonstrated when Codex immediately found a blocking bug in an onboarding flow that had survived for months.The Ironic Result✦ from: Claude Opus 5 produces excellent work but is one of the most frustrating models to interact with directly because it is timid, apologetic, and unusually dependent on human approval — refusing straightforward actions like resolving a one-line merge conflict because the code belonged to 'someone else's branch.'Root Cause: Humans Complete Forms Correctly✦ from: AI testing can be far more exhaustive than human testing because AI does not default to following the happy path, as demonstrated when Codex immediately found a blocking bug in an onboarding flow that had survived for months.User Perspective Reveals Hidden Pain✦ from: Persona testing with browser use — having AI use a product as a specific persona like a PM coming out of a meeting — can reveal structural friction that synthetic user research misses.

Related episodes

Video · a16z Podcast

Is Defense the Next Trillion-Dollar Category? | a16z American Dynamism Summit

→

Video · BG2 Pod

The SpaceX IPO, Fable 5, AI Capex Update & Market Check w/ Gavin Baker, Andrew Fox & Clark Tang

→

Article · Import AI

Import AI 464: Fables writes GPU kernels; AI automation; and analog computation

→