Any pattern that can be generated or found in nature can be efficiently discovered and modeled by a classical learning algorithm.
Demis Hassabis proposes that natural systems shaped by evolutionary processes have structure that can be learned by neural networks, making computationally intractable problems like protein folding solvable by classical systems.
transcript
Demis Hassabis: I felt that it's sort of a tradition, I think, of Nobel Prize lectures that you're supposed to be a little bit provocative, and I wanted to follow that tradition. What I was talking about there is if you take a step back and you look at all the work that we've done, especially with the AlphaX projects, so I'm thinking AlphaGo, of course, AlphaFold, what they really are is we're building models of very combinatorially high dimensional spaces that, you know, if you try to brute force a solution, find the best moving go, or find the exact shape of a protein, and if you enumerated all the possibilities, there wouldn't be enough time in the, time of the universe. So you have to do something much smarter. And what we did in both cases was build models of those environments. And that guided the search in a smart way. And that makes it tractable. So if you think about protein folding, which is obviously a natural system, you know, why should that be possible? How does physics do that? You know, proteins fold in milliseconds in our bodies. So somehow physics solves this problem that we've now also solved computationally. And I think the reason that's possible is that in nature, natural systems have structure because they were subject to evolutionary processes that shaped them. And if that's true, then you can maybe learn what that structure is. ... I sometimes call it survival of the stabilist or something like that, because, it's, of course, there's evolution for life, living things, but there's also, if you think about geological time, so the shape of mountains, that's been shaped by weathering processes, right, over thousands of years. But then you can even take it cosmological, the orbits of planets, the shapes of asteroids, these have all been survived kind of processes that have acted on them many, many times. So if that's true, then there should be some sort of pattern that you can kind of reverse learn and a kind of manifold, really, that helps you search to the right solution, to the right shape, and actually allow you to predict things about it in an efficient way, because it's not a random pattern, right? So it may not be possible for man-made things or abstract things like factorizing large numbers, because unless there's patterns in the number space, which there might be, but if there's not and it's uniform, then there's no pattern to learn. There's no model to learn that will help you search. You have to do brute force. So in that case, you maybe need a quantum computer, something like this. But in most things in nature that we're interested in are not like that. They have structure that evolved for a reason and survived over time. And if that's true, I think that's potentially learnable by a neural network.
provides context · 1