The loss landscape of neural networks undergoes a jamming phase transition identical to that of granular materials like sand: underparameterized systems are stuck in metastable states while overparameterized systems flow freely to flat minima.
Wyart draws a direct analogy between the physics of sand and machine learning: both are constraint-satisfaction problems with continuous degrees of freedom, and both exhibit the same jamming phase transition — the peak of the 'double descent' curve corresponds to the critical point where the system transitions from a rough, trapped landscape to a smooth, flowing one. ✦ AI generated
Matthieu Wyart · Machine Learning Street Talk · 2026-08-10 · original ↗
starts at this moment · 5:11
when you actually underparameterized when you don't have enough parameters you have a rough landscape with many metastable state and if you train your machine and you train it many times it will end up in different position where it's actually stuck but if you have enough parameters then suddenly the system can flow I in your landscape has many flat valleys and you can which have essentially zero energy. So there is really a close analogy we disco we discovered that like 9 years ago at the same time others find a very similar I mean the same phenomenon and called it double descent. So now this name has stuck but this double descent this peak of the double descent is really for physicists a jamming transition.
verbatim transcript · starts at 5:11
5:11are very intrigued by what is the geometry of this landscape and so what we discover is actually that this landscape has exactly the same phase transition as send it means that when you actually underparameterized when you don't have enough parameters you have a rough landscape with many metastable state and if you train your machine and you train it many times it will end up in different position where it's
5:34actually stuck but if you have enough parameters then suddenly the system can flow I in your landscape has many flat valleys and you can which have essentially zero energy. So there is really a close analogy we disco we discovered that like 9 years ago at the same time uh others find a very similar I mean the same phenomenon and called it double descent. So now this name has
6:02stuck but this double descent this peak of the double descent is really for physicists a jamming transition. So yeah, so so that brought me to machine learning and just maybe to uh to finish with that I mean we uh in the last four years we've been very much interested in another landscape that I think is even more interesting. It's a landscape of data. Uh so if you think
6:26about you know an image let's call X an image it's a vector you could ask what is the density of those images row of X and this question relates to what is the structure of the world and we think it's key to actually understand how a machine work >> does it make sense to talk about because obviously you know you're a physicist and you're applying this lens of