Data◆Article

DeepSeek V4-Pro's combined CSA/HCA compressed attention, which shortens the KV cache along the sequence dimension rather than per-token, cuts single-token inference FLOPs to 27% and KV cache size to 10% of DeepSeek V3.2 at a 1M-token context.

DeepSeek V4 alternates a milder sparse compression (CSA) with a much heavier dense compression (HCA) that shrink the KV cache by summarizing groups of tokens, delivering dramatic reported efficiency gains over DeepSeek V3.2 at million-token context lengths. ✦ AI generated

Sebastian Raschka · Ahead of AI · 2026-05-16 · original ↗

Share card ↓Export .md Export .txt

The DeepSeek V4 paper reports that, at a 1M-token context length, DeepSeek V4-Pro uses only 27% of the single-token inference FLOPs and 10% of the KV cache size compared with DeepSeek V3.2, which uses MLA and DeepSeek Sparse Attention (DSA).

Read full article ↗excerpt · fair-use quotation

✳ CSA ✳ DeepSeek V3.2 ✳ DeepSeek V4 ✳ HCA

Around this claim

In practice · 1

Optimizing inference requires balancing cost-per-token against architecture choices, using memory technologies like KV cache and CXL plus intelligent routing like NVIDIA's Dynamo to match workloads to the right GPU, CPU, or LPU.Thierry Pienaar · The TWIML AI Podcast · conf 70%

Context · 1

The expense of long-context generation comes less from holding the cache and more from sweeping through all of it on every single decoding token — the cache is a bandwidth cost as much as a storage one.Author · ByteByteGo Newsletter · conf 80%

This moment responds to

gives example → Optimizing inference requires a cost-per-token and architectural strategy, combining memory technologies like KV cache and CXL with intelligent routing tools like NVIDIA's Dynamo to balance workloads across GPU, CPU, or LPU types.Kaushik Shirhatti · The TWIML AI Podcast