The equivariant geometric attention (IPA) in AlphaFold 2 got most of the public credit for its success, but ablation studies showed it accounted for only about 2.5 of the 30-point accuracy gain over AlphaFold 1 — the real gains came from many smaller ideas stacked together.
Jumper recounts how, despite the field crediting geometric equivariance for AlphaFold 2's breakthrough, ablations showed it contributed only a small fraction of the gain — the real story is dozens of smaller, cumulative engineering wins. ✦ AI generated
John Jumper · Machine Learning Street Talk · 2026-06-22 · original ↗
starts at this moment · 29:46
“Can we talk through the predictive architectures of of the different versions of AlphaFold?”
My favorite review of of AlphaFold 2, we got the reviews back when we submit the paper. And one of them said, "This is six or seven papers worth of ideas." Right? And I think I think that was that was right. There are many many ideas that added up to be a transformative system. And many, you know, to use a baseball analogy, it's not one or two home runs. It's, you know, 18 doubles.
verbatim transcript · starts at 29:46
29:44to it. And I think what it really happens is I think we we were talking about it kind of at dinner last night at this AlphaFold dinner, but the equivariance is one like global SE3 symmetry is not a very powerful symmetry. It's not nearly as kind of big and powerful as a symmetry like, "Oh, all the residues are permutation invariant." Right? So, we do still have permutation
30:07uh invariant as probably the big symmetry of AlphaFold, right? We have a transformer that is position is relative position coded only. We clipped the relative position codings. But, I think this particular symmetry it's not like physics where you write down the symmetry group and then you derive the laws of physics from your big symmetry group and you get the standard model. This is This is a symmetry of a
30:30messy real-world problem that probably doesn't pin it down so much. So, I think it's good, but we shouldn't obsess about one good thing. Or you can't You don't want to valorize things. My favorite review of of AlphaFold 2, we got the reviews back when we submit the paper. And one of them said, "This is six or seven papers worth of ideas." Right? And I think I think that was that
30:53was right. There are many many ideas that added up to be a transformative system. And many, you know, to use a baseball analogy, it's not one or two home runs. It's, you know, 18 doubles. Right? That it it's really, you know, these mid-size wins stack together and together make a transformative system. Now, we would sometimes find in our ablations, we ran a double ablation, I think it was no recycling and no IPA. We
31:17turned off two things and performance cratered. Right? And I think it was kind of there are many problems we need to dissolve. We solved most of them two ways because it was better than solving one. And if you knock out both things, then your building maybe collapses. Or this was maybe a 12 or 15 point, which was our biggest ablation, which was still only half the gap to AlphaFold 1, right? We
31:38I remember doing the ablations and saying, "Guys, we've never crossed AlphaFold 1 performance." But a lot of those ablations actually went into AlphaFold 3. So, we said, "Okay, well, equivariance isn't super important." Um we had another ablation uh that if we take out, you know, giving the raw genetic information and give the pairwise correlations, that's one or two worse. So, maybe this fact that we're processing these all the time is not so