Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI

Andrej Karpathy: Just I think everything, so many things, even if they don't work, I think to a large extent you feel like it's a skill issue. It's not that the capability is not there. It's that you just haven't found a way to string it together of what's available. Like I just don't, I didn't give good enough instructions in the agent's MD file or whatever it may be. I don't have a nice enough memory tool that I put in there or something like that. So it all kind of feels like a skill issue when it doesn't work to some extent.

Andrej Karpathy: I think there's this sense that these apps that are in the app store for using these smart home devices, et cetera, these shouldn't even exist kind of in a certain sense. Like shouldn't it just be APIs and shouldn't agents be just using it directly? And wouldn't it like, I can do all kinds of home automation stuff that any individual app will not be able to do, right? And then LLM can actually drive the tools and call all the right tools and do pretty complicated things. And so in a certain sense, it does point to this maybe there's like an overproduction of lots of custom bespoke apps that shouldn't exist because agents kind of like crumble them up. And everything should be a lot more just like exposed API endpoints. And agents are the glue of the intelligence that actually like tool calls all the parts.

Andrej Karpathy: You experiment on the smaller models, you try to make it as autonomous as possible, remove researchers from the loop. They have way too much. What is the opposite? Yeah, they don't know. They shouldn't be touching any of this, really. And so you have to rewrite the whole thing, because right now, I mean, certainly they can contribute ideas. But okay, they shouldn't actually be enacting those ideas. There's a queue of ideas. And there's maybe an automated scientist that comes up with ideas based on all the archive papers and GitHub repos, and it funnels ideas in. Or researchers can contribute ideas, but it's a single queue, and there's workers that pull items, and they try them out. And whatever works just gets sort of put on the feature branch, and maybe some people monitor the feature branch and merge to the main branch sometimes.

Andrej Karpathy: You can imagine that different program.mds would give you different progress. So basically every research organization is described by program MD. A research organization is a set of markdown files that describe all the roles and how the whole thing connects. And you can imagine having a better research organization. So maybe they do fewer standups in the morning because they're useless. And this is all just code, right? And so you can, so one organization can have fewer standups, one organization can have more. One organization can be very risk-taking. One organization can be less. As you can definitely imagine that you have multiple research orgs, and then they all have code. And once you have code, then you can imagine tuning the code. So 100%, there's like the meta layer of it.