MechanismArticle
The scheme is a BLOB history column holding a compressed JSON array of all previous documents, plus an uncompressed parallel JSON array of timestamps.
The concrete design: a single table with a compressed JSON text-array BLOB and an uncompressed JSON array of Unix integer timestamps. ✦ AI generated
The author (as spoken to GPT-Live) · Simon Willison's Weblog · 2026-08-09 · original ↗
There is a history column on the single on this uh uh table and it's a blob, it's a BLOB so it stores binary data and then you just stick in there a Zlib or maybe even ZSTD um compressed JSON text array of all of the previous documents, and so you probably have two columns, right? You'd have a column that's this magic JSON array of text You have a second column which is a JSON array of timestamps and that doesn't need to be compressed at all, right? A timestamp can just be a uh it's an array of integers, right? Unix integers.
Read full article ↗excerpt · fair-use quotation
Around this claim
In practice · 2
The experimental prototype works extremely well: 1,000 simulated revisions of a document compressed from 20.4 MB of raw revision text to 80.3 KB as a Zstandard-compressed JSON array.The author (on GPT-5.6 Sol Pro output) · Simon Willison's Weblog · conf 80%The purpose of condense-json is to make it easier to store JSON that includes duplicated data from other related structures, which I use to save space in the SQLite logs generated by LLM.The author · Simon Willison's Weblog · conf 60%