Trained neural network weights can be treated as an input modality for training another neural network to analyze and generate new weights, analogous to how language models learn from text and image models learn from pixels.
Borth introduces weight-space learning, arguing that trained model weights are not just the output of training but can be used as input data for a new class of neural networks that analyze and generate weights, just as language models process text or vision models process pixels. ✦ AI generated
Damian Borth · The TWIML AI Podcast · 2026-07-27 · original ↗
starts at this moment · 2:49
Exactly. So if you think about machine learning, machine learning has this idea of you have data and uh some output um in classical supervised machine learning data and some predictions and you train neural network uh in between to mimic uh the data set mimic the distribution of the data set and uh the outcome during this very expensive training procedure is a set of weights a configuration of parameters that define the neural network like the DNA of the neuronet network. um this is classical machine learning supervised unsupervised self-s supervised that fuels a lot of innovation over the last 10 years and with geni you know moved to next stage if you look at what happened over the last couple of years more and and more and more of those models have been published uh publicly are online accessible at you know repositories like hugging face or github. So we basically thought about this very simple idea. What happens actually if we take the weights of trained neuronet networks as the input to train a neural network to understand these weights that we have out there much much better. So to take another analogy you you know language models you take a big model you train this on every single sentence on the internet at the end you have a language model able to analyze language and to generate language. You can take the same idea for pixels. You take a big model, you train on all the pixels on the internet and you can analyze pixels and you can generate pixels. We do the same idea on all the weights of trained neural networks. So we can analyze weights of neuronet networks and we can generate weights of neuronet networks. um as straightforward as it is obviously there's a little bit more into the details but thinking about that you can treat the ways as a input modality gives you suddenly this opportunity of thinking about okay what would be language translation in with more like neural network models right what would be generation of you know words and tokens that are words in generation of tokens that are weights and can we be much much faster in creating new weights for particular the task or can we be much more precise in analyzing weights when somebody gives me a new network that I'm not knowledgeable about and I never saw before and then you know you have this new entire world this you know empty space of things you can do with weights um that you know you kind of carry into the community and hope that there's somebody listening and continuing and you know building up a community which happened over the last two three years which is very exciting because there are more people about that And yeah uh weights are exciting uh not only as the output of learning but as the input for learning.
verbatim transcript · starts at 2:49
2:49much more that we can learn from these weights. >> Exactly. So if you think about machine learning, machine learning has this idea of you have data and uh some output um in classical supervised machine learning data and some predictions and you train neural network uh in between to mimic uh the data set mimic the distribution of the data set and uh the outcome during this very expensive training procedure
3:15is a set of weights a configuration of parameters that define the neural network like the DNA of the neuronet network. Um this is classical machine learning supervised unsupervised self-s supervised that fuels a lot of innovation over the last 10 years and with geni you know moved to next stage if you look at what happened over the last couple of years more and and more and more of those models have been
3:41published uh publicly are online accessible at you know repositories like hugging face or github. So we basically thought about this very simple idea. What happens actually if we take the weights of trained neuronet networks as the input to train a neural network to understand these weights that we have out there much much better. So to take another analogy you you know language models you take a big model you train
4:10this on every single sentence on the internet at the end you have a language model able to analyze language and to generate language. You can take the same idea for pixels. You take a big model, you train on all the pixels on the internet and you can analyze pixels and you can generate pixels. We do the same idea on all the weights of trained neural networks. So we can analyze
4:31weights of human networks and we can generate weights of neuronet networks. um as straightforward as it is obviously there's a little bit more into the details but thinking about that that you can treat the ways as a input modality gives you suddenly this opportunity of of thinking about okay what would be language translation in with more like neural network models right what would be generation of you know words and
4:56tokens that are words in generation of tokens that are weights and can we be much much faster in creating new weights for particular the task or can we be much more precise in analyzing weights when somebody gives me a new network that I'm not knowledgeable about and I never saw before and then you know you have this new entire world this you know empty space of things you can do with
5:20weights um that you know you kind of carry into the community and hope that there's somebody listening and continuing and uh you know building up a community which happened over the last two three years which is very exciting because there are more people about that And yeah uh weights are exciting uh not only as the output of learning but as the input for learning. >> You mentioned you started this effort in
5:442021. Where did you start from and then we'll work our way towards like where we are now with weights based learning. >> Originally this idea came 2020 and we got the first paper published 21 and the idea was very simple. Can we fingerprint a neural network or version a neural network like we can do with software? Right? In software you have you can do a diff right you have a 1 million lines of
- ·Trained neural network weights are like the DNA of the network
- ·Weights can be the input to train another neural network
- ·Treating weights as input modality opens a new learning paradigm
- ·Language models learn from every sentence on the internet
- ·Vision models learn from every pixel on the internet
- ·Same approach applied to all available trained weights
- ·New networks can analyze weights of unfamiliar models
- ·New networks can generate weights for specific tasks
- ·Weights become input for learning, not just output