Liquid AI built an automated, hardware-in-the-loop architecture search system (AFMD) specifically to eliminate human bias in architecture design, because even at top labs a small group of people is really just calling the shots based on personal intuition.

43:09you that why did we take this approach you know like to design an architecture? Why? because we wanted to remove all the human biases early on as we are actually like building architectures. One of the things that we realize like culturally at companies like this is what I can tell you like even at largest foundation model labs uh in in the US right now entropic and open AI there are a bunch

43:29of people like people that are coming from the science like I call them the avengers of the architectures or avengers of the you know like post training or or or let's say pre-training these groups of people there's usually like a very small set of people that are calling the shots on like oh you know what you're going to tweak this portion of this architecture so that it performs

43:50better. Why? Because in my personal experiences, it has started working better. If you're really truly think and this is something that is broken in all the foundation model labs. You cannot say that like somebody has a fix to this. But now the recursive self-improvement kind of process is actually fixing for that because now people are just finally realizing you got to give it to the algorithms. You

44:10know, you have to be bitter lessons. You have to you have bitter lesson people. So you got to you you you got to be giving it to a systematic way to actually find out what is the true architecture for the for the problems that you want to solve. You can build like a general purpose computer. The insights that I shared with you in in in in the format of the scaling laws of

44:30neural networks is coming out of our massive exploration of the space of architectures. You know the fact that in the smaller kind of category of models the some biases on the architecture help in the larger instances of the models you don't need to bias the systems you can actually go pure convolutions pure transformers and you can go pure let's say recurrences that are like very simplified you don't need to add any

44:56form of kind of you know specialized kind of treatment like gating there you know gated delta nets you know and then there's the mambas and there's the jambas and there's There's a lot of different variations of these architectures that are coming out. What you want to do though, you want to you want to be completely unbiased. Even us ourselves, you know, day one, like when we actually started liquid, we had one