DeepSeek V4's manifold-constrained hyper-connections (mHC) widen the residual stream into several parallel streams while forcing the mixing matrices to be doubly stochastic, making the wider residual pathway stable enough to scale to production with minimal overhead.

DeepSeek V4 productionizes manifold-constrained hyper-connections, replacing the single residual stream with several interacting streams whose mixing matrices are constrained to be non-negative and row/column-normalized, stabilizing the wider residual pathway at large scale. ✦ AI generated

Sebastian Raschka · Ahead of AI · 2026-05-16 · original ↗

In mHC, this residual mapping is projected onto the manifold of doubly stochastic matrices, meaning all entries are non-negative and each row and column sums to 1. This makes the residual mixing behave more like a stable redistribution of information across streams.

Read full article ↗excerpt · fair-use quotation