Video · 2026-07-31 · 1h 20m · 6 moments

How Decagon Runs 90% of Its Agents on Open-Source Models

✦ AI generated

timeline · colored by role

Claim◆

90% of Decagon's workflow runs on open-source models because smaller, fine-tuned models deliver lower latency and higher performance on the specific tasks of an agent's conversation, while actually outperforming large frontier models on those specialized tasks.

Jesse explains Decagon's migration from frontier models to 90% open-source: as the company scaled and launched voice agents, latency became critical, and frontier labs' small models couldn't be controlled and fine-tuned to their needs. Fine-tuned smaller models prove as good or better on individual agent subtasks.

transcript

Jesse: So today 90% of our workflow is on open source and um you know again the main reason was for latency to really optimize our voice agents and um I think we've just over the last year we've seen tremendous improvement in like how how it sounds how it feels and but still also like keeping the accuracy high um and then the remaining 10% of course we're still using the uh the closed source models and the frontier models for a lot of you know new new projects or new products

Claim

The trade-off between smart expensive models and dumb cheap models is false: a fine-tuned smaller model is better at the task, cheaper, and faster than a large state-of-the-art model on that specific task.

Jesse argues the common Twitter debate frames a false trade-off. In practice, fine-tuning smaller, dumber models yields all three wins on the specific task: better performance, cheaper, and faster.

transcript

Jesse: often times you know when you see these debates being had on Twitter the the trade-off tends to be oh do we want uh you know the smartest model that is very expensive or can we like dumb it down a little bit and get it cheaper. I actually think that is a false trade-off... even if you have a quote dumber model, you can get it to higher performance on that specific task. So when we fine-tune smaller, dumber models, it's that they're just not as general purpose, but on the specific task we want them to do, they actually outperform the large, smart, state-of-the-art models... So we end up getting all three things. It is better at the toss. It is cheaper and it is faster.

explains mechanism · 3

Prediction

Enterprises will eventually move to open-source post-training but more slowly than people think, because fine-tuning requires gathering data, building custom evals and benchmarks; open source becomes strictly better once a use case is solidified and in production at scale.

Jesse predicts enterprises will adopt open-source fine-tuning but slower than expected due to the difficulty of data, custom evals, and governance. He sees a pattern: new/experimental use cases start on frontier models (which is why open-source inference share is falling), but once a use case is solidified in production, there's strong economic incentive to switch.

transcript

Jesse: I think they'll get there but it'll probably take longer than people think because... even with our team fine-tuning these models is non-trivial... You have to get the data. And then more importantly, you have to like have good eval... it's strictly better to use open source models because when your use case is solidified and you're in production at scale and you're pretty sure this is the the sort of shape of the agent then there's no reason not to use open source because you you get these latency benefits and at the same time you get the cost benefits

provides context · 1

Context

Application-layer companies will always have a place because even with AGI, agents need somewhere to store work, pull information from, and reason about things; the 'labs are the last startups' narrative is incorrect.

Jesse pushes back on the dominant narrative that Anthropic and OpenAI are the last startups. Noting that humans are analogous to AGI yet still need databases and CRMs, he argues AI agents will likewise need software infrastructure, so application-layer companies persist—potentially becoming vertically specialized labs.

transcript

Jesse: I'm not as bought into the the labs are less startup view of the world... in a way we human beings are kind of AGI right and human beings have needed to use software for lots of things you know you need databases to put stuff in you need CRM to track things and I think even once you have AGI all our AGI agents are going to need somewhere to store work and pull information from and reason about things. So... I don't think software as a whole in any meaningful way is going away

Claim

Forward-deployed engineers are necessary now for early-stage AI companies because workflows are entirely new, but long-term companies should productize the workflows and stop relying on FDEs, otherwise they become glorified consulting firms.

Asha argues FDEs are being overused. They're necessary at early-stage AI companies because nobody knows the workflows yet—FDEs embed with customers to learn and pave them. But once the workflow is known, it must be productized; relying on FDEs long-term just produces a glorified consulting shop.

transcript

Asha: for deployed engineers are necessary or newly necessary for early stage AI companies because the workflows are new... with AI products nobody knows what the workflows are because nobody's used these things before so a for deployed engineer in this case is honestly just embedding with the customer to learn the workflow for the first time... uh but long term, I think they should just be building product... once you know what the workflow is, if you can productize it, you should productize it and then become the typical company with these scaling properties of a tech company. And if you can't do that, then you're just building a glorified consulting truck.

Claim◆

Decagon's moat is the ability to make models deployable within enterprises: the software, governance, testing, collaboration, and integration infrastructure built around them that labs and raw models don't provide.

Jesse argues that in the short term Decagon's moat is enterprise deployment infrastructure—the ability to constrain, test, govern, and integrate models into legacy enterprise systems. He acknowledges this may commoditize in a few years when agents can build it on the fly.

transcript

Jesse: I think in the short term actually it is the ability to work with enterprise resources... the capability of models today is far greater than they are being used for within the enterprise... to make models like this deployable within the enterprise you need to say okay I need a way to be able to tell the model what it can and cannot do... Then I need a way to make sure that hundreds of people within the enterprise can collaborate... Then I need a way to be able to test this model and make sure that it doesn't cross any like regulatory lines... and I think for the next few years that's probably going to be a you know the primary thing that these models need to be able to work. uh now once that gets commoditized because the agents can build that on the fly that I don't know

provides context · 2

Highlight slides

Decagon's Migration to Open-Source Models✦ from: 90% of Decagon's workflow runs on open-source models because smaller, fine-tuned models deliver lower latency and higher performance on the specific tasks of an agent's conversation, while actually outperforming large frontier models on those specialized tasks.Why Open-Source Won the Workflow✦ from: 90% of Decagon's workflow runs on open-source models because smaller, fine-tuned models deliver lower latency and higher performance on the specific tasks of an agent's conversation, while actually outperforming large frontier models on those specialized tasks.Decagon's Moat: Enterprise Deployment Infrastructure✦ from: Decagon's moat is the ability to make models deployable within enterprises: the software, governance, testing, collaboration, and integration infrastructure built around them that labs and raw models don't provide.Why the Moat Is Temporary✦ from: Decagon's moat is the ability to make models deployable within enterprises: the software, governance, testing, collaboration, and integration infrastructure built around them that labs and raw models don't provide.

Related episodes

Video · All-In Podcast

OpenAI vs Anthropic IPOs, Anthropic $3T, Zuck's Price War, China Ends Open Source?, Trump Accounts

→

Article · Interconnects

Teaching Everyone to Fish for Tokens

→

Video · BG2 Pod

The SpaceX IPO, Fable 5, AI Capex Update & Market Check w/ Gavin Baker, Andrew Fox & Clark Tang

→