The inference business for open-weight models is an enormous and rapidly growing opportunity — Fireworks is doing over $1B ARR in 3.5 years with expanding gross margins — and this is where the real AI money is being made, not in applications.
Rory breaks down the three layers of AI spending — infrastructure ($800-900B), foundation models (~$100B), and applications (barely $40-50B combined) — arguing that inference providers like Fireworks are in the sweet spot of exploding open-weight demand. ✦ AI generated
Rory · 20VC · 2026-07-23 · original ↗
starts at this moment · 31:04
My mental model is I divide the AI world up into three buckets. It's the making AI, the infrastructure layer, right? And you're right, the spend there is 8 $900 billion a year. Then there's the two foundation model companies themselves and they're doing plus or minus hundred billion dollars a year, right? And then taking those guys out, rounding up every other apps company, right? You struggle to make 40 or 50 bill. You struggle. You start with cursor at four because I think coding is an app. You know, you by the time you're throwing in Harvey, you're adding two 300 million, right? It's amazing. I mean, just the difference in spend. And you know, at some point, the people spending a trillion dollars a year are going to want some apps to pay for all this, right? But right now the volume has it's been front end loaded on the infrastructure side.
verbatim transcript · starts at 31:04
30:47point. It might be now. It spaces commodify. It doesn't kill everybody, but it might maim you, but it also makes it attractive. It also makes you attractive to acquirers for a window and then that window closes. The commod the commodification window closes. And um probably why Curser wasn't dumb to sell at 60 billion. I think we can agree that's true. I don't love the commoditization description. I
31:10think it it it it's an overloaded term, but yes, I >> well an included feature and more and more folks will include your function some version of your functionality in their product, right? >> That's that that's exactly it. Where Yeah. which segus to the next topic, >> which is fireworks. >> Yeah. And inference in general. Yeah. I mean, >> take it away, Rory. I'm going to butcher
31:32whatever context that you want to take it on. >> No, no, you do. No, you do first cuz I'm I'm just >> Are you sure? I'll lay the framework and then you can just destroy it. Steamroll away. >> Fireworks uh a leading inference provider uh announced their latest round which was a $1.5 billion round done by um uh index Gavin Baker uh Nvidia Lightseed 20VC. Amazing firms. Um
32:02they're incredible. Uh thank you very that they're really [ __ ] good. Lynn is amazing. Uh doing over a billion in AR got there in three and a half years and they announced around 40 trillion tokens a day up from 15. I mean I think the story is inference. Yes. First of all I agree. Yeah inference is a hu it goes back ironically to the prior comment on openweight models. it this
32:25kind of standalone inference is a big business right and you know obviously you know inference is both something that's done within the the frontier model companies where they do their own inference and people like Microsoft and Google provide the capex and provide the compute for that but people like fireworks and base 10 they and file they all make their money offering a variety of these open openweight models to third
32:49party developers and enterprises that want to use open source models to do AI, right? And it, as I said, the two trends go together. They're exploding because the open source trend is exploding. So if you're base 10, if you're foul more media, if you're fireworks, if you're together, this is your marker than your moment, right? So yeah, I this is cuz this is how you access those because I
33:14can tell you one thing. We can going back to the discussion about open weight models from China. It's one thing to decide to use an openweight model on fireworks in the US. What you're not going to do is be using the API back to China even if they'd let you. Right? So this is a onetoone linkage between the open source the open way trend. Right? These are the
33:35companies that are benefiting massively from that trend and it's not the only kind of route for inference. There are you know inference for US-based models etc etc. But the vast bulk of it is oh my god I'm sourcing Quen. I'm sourcing Kim. I'm hosting Quen Kimmy. I want to use it as Cursor. I want to get someone to provide me some inference. These guys exist and you know they have lots of