Article · 2026-07-15 · 6 moments

Context engineering with Dex Horthy

Dex Horthy explains why context engineering is key to building more effective AI-assisted software without sacrificing code quality. ✦ AI generated

Read original ↗

GUEST Dex Horthy HOST Gergely Orosz

Mechanism

Coding models are trained in ways that make codebases worse over time, because they're optimized to win SWE-bench-style benchmarks that reward known fixes but can't measure bad architecture.

Dex argues LLM-driven codebase degradation stems from benchmark incentives: SWE-bench-style evals reward reproducing known fixes but have no way to penalize poor architectural decisions.

transcript

Dex Horthy: Dex believes that the reason we see LLMs “degrade” existing codebases is because they are optimized to do well on SWE-bench-style benchmarks. These benchmarks reward reproducing a known fix in codebases like Django, but cannot measure poor architecture decisions.

Mechanism

Compressing a long, noisy context into a Markdown document and starting a fresh session pointed at that compressed context improves outcomes on complex projects.

Dex's 'intentional compaction' workflow chains sessions: one reads code and emits a research doc, the next turns tickets into a design doc, and a final one turns both into a plan, with humans reviewing the design step.

transcript

Dex Horthy: He will take a long and noisy context, compress it into a Markdown document, then start a new session fresh, pointing the model to this “compressed context” that is in the Markdown.

Claim

Investing planning time in high-leverage areas like design and architecture, then letting agents generate code without reviewing all of it, can make teams move 2-3x faster than fully manual coding.

Of three 'software factory' approaches, Dex favors finding leverage points where an hour of planning saves four hours of implementation, keeping humans focused on design and key decisions while agents handle code generation.

transcript

Dex Horthy: Find leverage, but keep people in the loop. Find out where an hour spent in planning could save four hours’ worth of implementation, in terms of fewer bugs. Invest more time in areas with leverage: design, architecture, and key decisions. Then, let the agent generate code and don’t insist on reviewing all of it.

Data◆

Model performance degrades well before a context window is actually full, so usable context should be capped far below the stated maximum.

Dex's heuristic: push a 1M-token model to only 300-400K tokens, and smaller models to about 100K, before hitting a 'dumb zone' where the model starts making careless mistakes like deleting files.

transcript

Dex Horthy: For a model with a 1M context window, Dex pushes it to around 300-400K when it feels right. For smaller models, he stops at around 100K. You hit the “dumb zone” when its performance starts to degrade because the context window fills up beyond this heuristic limit, and the model begins doing increasingly stupid things like deleting your .env file, for example.

Anecdote◆

Shipping AI-written code without human review leads to production disaster within months.

Dex describes a July 2025 experiment where his team let a model write code with zero human review; four months later production broke in a way even Opus 4.1 couldn't diagnose, forcing the whole system to be scrapped.

transcript

Dex Horthy: Dex experimented with having the model write the code and humans not reviewing anything in July 2025. Four months later, they shut things down and threw the whole system out. Production broke, and no matter how much the team prompted Opus 4.1, the model could not find the root cause.

Mechanism

When a model starts saying things like 'you're completely right,' the session has become trajectory-poisoned and should be abandoned rather than continued.

Because models are autoregressive, once a session falls into a loop of mistake-then-apology, it keeps predicting more mistakes, so Dex treats certain stock phrases as a signal to start fresh.

transcript

Dex Horthy: “You’re completely right!” or “you’re right to push back on that” are phrases that mean it’s time to start a new session. These responses mean the LLM session is trajectory-poisoned, and you’re wasting time and tokens to continue.

Highlight slides

The Zero-Review Experiment✦ from: Shipping AI-written code without human review leads to production disaster within months.The 'Dumb Zone': Context Limits Are Lower Than Advertised✦ from: Model performance degrades well before a context window is actually full, so usable context should be capped far below the stated maximum.Four Months Later: Collapse✦ from: Shipping AI-written code without human review leads to production disaster within months.Dex's Usable Context Heuristic✦ from: Model performance degrades well before a context window is actually full, so usable context should be capped far below the stated maximum.

Related episodes

Article · Import AI

Import AI 464: Fables writes GPU kernels; AI automation; and analog computation

→

Article · Latent Space

[AINews] Thinky's Inkling: 975B-A41B multimodal, new best American Apache 2.0 open model (with Inkling-Small, 276B-A12B)

→

Article · Import AI

Import AI 465: Open vs closed gaps; Kimi K3; Demis' big policy plan

→