The newest generation of models is good enough to run as orchestrators that never do work directly, instead spawning cheaper sub-agent models in parallel for exploration and code changes, which cuts cost while preserving the primary model's intelligence.

22:46if you switch models in the middle of a session, that new model is like a full fresh cache bust and it's like a very expensive switch. >> Yeah. >> Um so I think it's hard at that layer. The other side that we are very interested in, um especially with some of the there's like there's like next generation of models, Uh they are actually very good at this

23:05orchestrator pattern. I think previous models I think people tried this with previous models that I don't think they were good enough for the average person. But some people on our team with some of these newer models they've set it up where their primary session is this expensive model but it's prompted to never actually do anything. It's prompted to only spawn sub agents for everything. And the sub agent is a

23:25cheaper model. And this makes way more sense I think because the agent is kind of choosing like you can kind of gradually use a cheaper model for like exploring doing much of code changes etc. But kind of still have the intelligence of the primary model. And I think in that this ends up being cheaper and it's also kind of like these newer models are very good at parallel

23:45work. So like you can like in a single session be working on lots of different things with background sub agents that then kind of finish and and wake up the primary one. And it feels pretty good cuz you're just in a single session. Yeah. I think that's the version of model routing that makes sense for me. >> And you said a couple minutes ago like you're burning

24:01a lot of tokens and using some models that are not available yet. What are those? What are you What are you using? What do you got access to? >> It's very It's very unclear what I'm allowed to say and what I'm not. Like >> [laughter] >> I posted stuff that I thought was okay cuz I've seen other stuff posted by other people post about it and they told

24:16me it was not okay. So just assume that we have access to obviously like the Both Open AI and Anthropic have a pretty big preview program that they give to people for feedback. Mhm. So you know we we kind of see things a little bit early. And the latest model from I'm not going to be specific. One of these labs is the one that caused us to

24:41go 5x on on our usage. Uh so >> Really? So Dex, who do we need to contact to get Syntax pre-access to these? >> I don't have no idea. >> [laughter] >> He's not saying. >> I don't believe that. >> let me dig into that a second. One of these unreleased models has caused you to go 5x on your token usage and that is because that's not because it's more

25:05token hungry, that's because it's changed the way that you're >> is addicted to this thing, yeah. And and I want to preface this by saying like, you know, for people that aren't aware, we're a very conservative team. We've like for years been pretty conservative about coding with AI. We're like very much not the AI psychosis people. We've been pretty measured about how we use AI and we what