The Jacobian lens is an unlearned, purely algorithmic transform applied to a model's intermediate-layer residuals that reveals a well-defined workspace of concepts, letting researchers identify and perturb specific concept directions in the middle layers.

28:32I might need some handholding here so bear with me. Um a as I understand it the Jacobian space is essentially a kind of mathematical framework of the residuals. So what they've done is they've perturbed the model and they spotted the residuals which react to certain things and when they accumulate all the residuals and they transform it they find that there's a well- definfined kind of space of concepts and

29:03those concepts are in the mid part of the um of the transformer structure I guess. Um so you have the workspace is basically in the middle part the intermediate processing stage that's what they call it. It's the intermediate depths and they find also that you can find concepts on multiple layers. So you have like layer 50, layer 71, layer 99. So as it proceeds through the intermediate

29:36processing, it kind of converges on an answer. So one one thing that they do is for mental arithmetic, they show layer 58. It identifies it as math. Layer 75 identifies the first part of the equation inside the brackets as 21. Layer layer 83 [clears throat] it decides the next part of the equation which is bracket times a number and layer 99 it gets the answer kind of at

30:07at the 99 layer. So as I understand it that's that's part of uh the Jacobian basically shows the progression of the thought process uh from layer to layer and having done that have they identified actual so you're looking at this broad matrix right it's basically like millions of numbers in in multiple dimensions it's a hyperdimensional space and And the jacobian is basically a transform on top of that hyperdimensional space. Right?