Companies progress through a maturity cycle from relying entirely on per-token closed model APIs, to hyperscaler provisioned-throughput deals, to dedicated inference providers or in-house infrastructure as their AI product scales.
Kiely lays out a product-maturity progression (not company-size based) from starting on closed-API providers, to hyperscaler commits when cost or capacity problems hit, to dedicated inference providers or in-house/edge deployment. ✦ AI generated
Philip Kiely · The TWIML AI Podcast · 2026-04-30 · original ↗
starts at this moment · 23:55
“What are the dominant models or the dominant setups that you're seeing and like what drives a company to go from one to the next?”
Option number one is to rely entirely on per token closed model providers. And everyone starts here just about. Everyone should start here. It's really easy. You get frontier intelligence with an API key. And that's a hard thing to beat when you're starting out. I think the next level often looks like running into one of two problems, either a cost problem or a capacity problem.
verbatim transcript · starts at 23:55
23:55few options. Option number one is to rely entirely on per token closed model providers. And everyone starts here just about. Everyone should start here. It's really easy. You get front to you intelligence with an API key. And and that's a hard a hard thing to beat when you're starting out. I think the the next level often looks like running into one of two problems, either a cost problem or a capacity problem.
24:25And saying like either my my token costs are just getting out of control and or I just can't get enough tokens to do the thing I want to do. And for that often times people start turning to hyperscalers, AWS, GCP, et cetera, and doing things like provisioned throughput purchases, doing things like spinning up models on like a bedrock or a vortex or an Azure AI foundry or something like that. And this
24:53kicks the can down the road a little bit because now you have like a large scale commit with a provider who has of course massive scale and the ability to to deliver you some capacity. And then the sort of next step that I see a lot of times is going on to more of a dedicated inference provider like a Base 10, where companies have, you know, the the models with the weights that
25:18they own and they start to set up, you know, a a specialized deployment. I think the sort of other fork in the road there is to go for something that's a little bit more in house, either, you know, building an in house platform, staffing that team up and and trying to deliver a a sort of best in class inference platform internally. Uh or, you know, in some cases like
25:40going really deep into edge inference, especially, you know, if if you want to do a lot of sort of field work, you you replace all of the distributed systems problems with internet of things problems, but you you have a a very similar challenge in scaling inference for sort of a large distributed edge network. I see a lot less of like companies going out and making enormous capital purchases of