The first version of turbopuffer was the simplest possible thing — barely reading the literature on log-structured merge trees, just running a clustering algorithm on vectors and putting clusters in files on S3.

Simon describes building the initial version of turbopuffer in summer 2023 with minimal complexity: running a clustering algorithm on vectors, organizing results into files on S3, and using NGINX as a caching reverse proxy — without even implementing a dedicated caching layer. ✦ AI generated

Simon Eskildsen · The Pragmatic Engineer · 2026-07-21 · original ↗

I just became fully obsessed that summer [of 2023] with building it. The first version was the simplest possible thing. I'm a very pragmatic person, so I didn't get buried in detail. I barely read the literature on log-structured merge (LSM). The simplest way to do this is to run a clustering algorithm on the vectors. You get the clusters, and then put the clusters in files. The files are called 'cluster_01', 'cluster_02', 'cluster_03.'

Read full article ↗excerpt · fair-use quotation