The analogy between jamming of sand and the loss landscape of machines is direct and valid because both are systems of continuous degrees of freedom trying to satisfy constraints, and satisfiability problems with continuous degrees of freedom fall into a universality class.
Wyart defends the coherence of applying statistical physics to machine learning: in both sand and model training you have continuous degrees of freedom collectively satisfying constraints, which forces a universality class. The analogy is about the algorithm (gradient descent flowing down an energy landscape), not the material substrate. ✦ AI generated
Matthieu Wyart · Machine Learning Street Talk · 2026-08-10 · original ↗
starts at this moment · 7:40
“does it make sense to talk about because obviously you know you're a physicist and you're applying this lens of analysis to large language models and like naively I'm looking at this and saying well that there's it doesn't feel like a material substrate it doesn't feel like it has the same type of dynamics as things do in the real world. But but indeed when we look at the training dynamics of LLMs and when we look at the types of representations they learn we we could adopt a physics lens and say there are course grainings and there are phase changes and and and whatnot. Do you think it's coherent to make that analogy?”
for the specific example I gave you about the sand and uh and uh and the loss landscape of of machines I think the analogy is is is very direct in this case because in both cases what you have are essentially degrees of freedom. In one case those are the part particle of sand in the other case there are the parameters of your large model and in both cases the system are trying to satisfy constraints. Um so for for sand essentially the particles are just trying to avoid each other but for the parameters what they are trying to collectively do is to fit data. So the more data you have the more constraint you have and at the end the universality there is what we've argued from physics but applies there that if you have a problem of satisfiability of constraint and you have continuous degrees of freedom that can change continuously then boom you have a universality class. So in this sense yes there's something universal about those kind of problems
verbatim transcript · starts at 7:40
7:30first to propose that light was a wave. How did he propose that? I mean he noticed that waves on the ocean could cross each other without interacting and he noticed it was the same for of light and so he made this analogy. I think it's even hard for me to to talk about because I think it's so fundamental that we are always you know building our understanding in terms of analogies. Um
7:53so for the specific example I gave you about the sand and uh and uh and the loss landscape of of machines I think the analogy is is is very direct in this case because in both cases what you have are essentially degrees of freedom. In one case those are the part particle of sand in the other case there are the parameters of your large model and in
8:16both cases the system are trying to satisfy constraints. Um so for for sand essentially the particles are just trying to avoid each other but for the parameters what they are trying to collectively do is to fit data. So the more data you have the more constraint you have and at the end the universality there is what we've argued from physics but applies there that if you have a
8:39problem of satisfiability of constraint and you have continuous degrees of freedom that can change continuously then boom you have a universality class. So in this sense yes there's something universal about those kind of problems and so but that's a very I that's very specific example I don't want to say that everything is always the same but but this specific problem of jamming of sand and the one of you know love
9:04landscape of machines is very much the same yes >> it's such a tantalizing idea because I think it is constraints all the way down and in evolution we have um like naturally convergent patterns, reoccurring patterns like carsonization. And I I I guess the only critique to this is it feels like in neural networks or just in computers, we don't have the same kinds of physical constraints. You
9:31know, we we don't have, you know, two objects can't touch each other at the same time and the laws of physics and and so on. So the the constraints are there by dent of, you know, statistical patterns in the data, but they still apply pressure on the training process. Is are those still valid constraints? >> Yes. So here I was really not talking about any sort of constraint of the