ZAYA1-8B's Compressed Convolutional Attention performs the attention operation directly inside a compressed latent space, rather than compressing only the stored KV cache as MLA does, which reduces both KV cache size and attention FLOPs.

Zyphra's ZAYA1-8B uses Compressed Convolutional Attention, which compresses Q, K, and V and computes attention directly in that compressed space, with added convolutional mixing on Q/K to preserve local context that compression would otherwise lose. ✦ AI generated

Sebastian Raschka · Ahead of AI · 2026-05-16 · original ↗

CCA compresses Q, K, and V and performs the attention operation directly in the compressed latent space. This is why CCA can reduce not only KV cache size, but also attention FLOPs during prefill and training.

Read full article ↗excerpt · fair-use quotation