When reviewing a project, benchmarks are unsatisfying because napkin math shows you what the theoretical lower bound should be, and a large gap means either the benchmark is wrong or your understanding is flawed.

Simon explains how he used napkin math to challenge product teams' benchmarks: if a query should take 10 milliseconds by first-principles calculation but the benchmark shows 10 seconds, something is wrong — often the benchmark is measuring a distributed query across many nodes. ✦ AI generated

Simon Eskildsen · The Pragmatic Engineer · 2026-07-21 · original ↗

When reviewing a project, a product team would tell me that they chose Database A over Database B because they benchmarked both, and Database A was better. I hate benchmarks because making design decisions based on benchmarks is not a satisfying answer to me. For example, you're saying that with Database A, as per the benchmark, it takes 10 seconds to do this one thing. But it should take 10 milliseconds if you do the napkin math.

Read full article ↗excerpt · fair-use quotation