Black sparse featurizers are a generalization of sparse autoencoders in which each feature is a small vector/network rather than a single scalar, recovering semantically meaningful subspaces unsupervised — in some cases the recovered activation structure itself tracks the 3D structure of the represented object (like a walking wolf's body and tail).

33:46kind of super long sparse vector, now each of those little positions in this like very long concept vector is itself a little network. And because of that, now we have room for kind of a richer representation of concepts, but the kind of same trick of like localizing concepts to individual spots on the on the super big sparse thing is the same with just this kind of additional enhancement that now allows

34:19you to have again richer representations and you can look inside for like geometries even within these little blocks. I think the easiest way to think about it is yeah like it's a generalization of an SAE where an SAPE assumes that features are onedimensional and instead you just don't have to do that. So instead of a scaler for every feature you can have a vector for every feature and some tricks in the machine

34:41learning that you can do in order to make sure that this like trains correctly and it and it learns. But what we find on various models is that this is successful in in recovering in an unsupervised way semantically meaningful subspaces and we find that features are much richer than we may have otherwise seen and they don't suffer from some of the same pathologies sapes suffer from. So in image models, I think we have a

35:05bunch of great examples where like maybe you'll find like a rabbit feature, right? And if an SAE collapse that to a single dimension, a BSF can represent it as a few dimensions. And you find that within those few dimensions, you have rabbit ears up here and then you have the rabbit face over here. And what some of the really remarkable things if you go and like look at some of the examples

35:26we've put out is you can sometimes see the like 3D structure of the thing that's being represented for image models for instance in the structure of the activations itself that's recovered in this unsupervised way cuz often coordinates are like represented like within these spaces as well. So there's one like great example where we have this like a beautiful GIF of it's a wolf that's that's walking and you look at

35:50the as it's walking its body is kind of wiggling and its tail is shaking and you look at the activations that were recovered in the unsupervised subspace and you see it just like wiggling in the video as the dog is wiggling kind of reflecting that that actual structure showing that the model's like tracking this like particular object. It has semantics along the different parts of the object and it's also tracking that

36:15object in in 3D space and it's doing all of that within a particular sub. I wonder like so I've been obsessed with this Graham technique that I'm sure you're familiar with that AE Studio put out with anthropic not too long ago and regular listeners know I've brought it up a bunch of times, right? The idea is simply if we start with some labeled data and we control where the gradients

36:42go in terms of like only allowing certain experts to be updated for certain kinds of data early in the training process. Then you get the great benefit of even for unlabeled data those data points gradients also tend to flow toward those same experts and there's sort of this absorption effect. And then the great promise of course is like or the great hope and promise is that you