Mechanism◆Article

Gemma 4's E2B and E4B models share KV cache tensors across layers, cutting KV cache memory by roughly half in long-context settings.

Gemma 4's small models reuse KV projections from earlier layers instead of recomputing them in every layer, roughly halving KV cache size and saving several GB of memory at 128K context. ✦ AI generated

Sebastian Raschka · Ahead of AI · 2026-05-16 · original ↗

Export .md Export .txt

Since we share roughly half of the KVs across layers, we save approximately half of the KV cache size. For the smallest E2B model, this results in a 2.7 GB saving (at bfloat16 precision) in long 128K contexts, as shown below.

Read full article ↗excerpt · fair-use quotation

▦ Google ✳ cross-layer attention ✳ Gemma 4

Related moments

Data◆Article

DeepSeek V4-Pro's combined CSA/HCA compressed attention, which shortens the KV cache along the sequence dimension rather than per-token, cuts single-token inference FLOPs to 27% and KV cache size to 10% of DeepSeek V3.2 at a 1M-token context.

Sebastian Raschka · Ahead of AI

MechanismArticle

Gemma 4's per-layer embeddings give each transformer layer its own small token-specific vector, adding representational capacity without scaling the whole transformer stack to a larger parameter count.

Sebastian Raschka · Ahead of AI