How Capital is Powering the AI Infrastructure Buildout with Magnetar Capital Managing Director Neil Tiwari

Neil Tiwari: And so in simple terms, when you have debt, you have principal and interest and you have to pay it off over time. And in these structures, typically the payback period on the CapEx was roughly 2 to three years. And the structures themselves, the debt was over 5 years, four to five years in length. where the entire debt amortized during the outstanding period that the debt was out. And so at the end, you ended up with 0 balance for the debt, and there was no balloon payment or anything that was really due on the back end. And so the question that often comes up is, isn't that a very risky type of structure, because these things are depreciating incredibly quickly. So I think there's two comments here. First is, On that depreciation question, in these kind of debt structures, it doesn't really matter because the debt's fully paid off by the end of the debt term against committed contractual contracts from investment-grade counterparties. And then at the very end, the actual upside or residual value, and I know there's a lot of questions on residual value, is held by, you know, the cloud player in this example, right, Court Week, right, or, you know, any others.

Neil Tiwari: I think, the way we see it and frame it really has to do with the demand signals and who are the eventual buyers and how is this being used. And so, at least from what our perspective, we continue to see insatiable demand. And if you go back to, the previous kind of big tech build out back in the early 2000s, There's obviously a lot of fiber that was being built and you had dark fiber, and an overbuild happening. And I think what you see here is I've, you know, you don't see any dark GPUs, any GPUs used. And then #2, you're starting to see actual economic value. So I think last year, enterprise AI had about 37 billion of total TAM, and it's continued to grow like crazy. And at least personally, and I'm sure you see this too, but I use these tools all the time and I find it incredibly valuable, right? The actual tokenomics of positive ROI is actually here now, I think from our perspective.

Neil Tiwari: Inference is a lot more complex than I think initially thought. And what I mean by that is it's not as simple as you train a model and then it's easy to inference it. In certain cases, you can do that on similar infrastructure, but there are issues around latency, fungibility of that, and really optimizing the cost of your compute on the inference side. How do you manage peaks of inference demand, and obviously it's not linear like training, your GPUs are on all the time, 100% of the time. And so with inference, you have a lot more variability. And so there's a lot more nuances in optimizing inference. I think the second thing that's observed that I've seen is inference is definitely a memory problem, a memory throughput problem. on the inference side, you have these kind of phases called pre-fill and decode, right? And how you optimize that across a fleet of GPUs is actually a unique technical problem. And then the third is what I would say is distribution. You know, a lot of times training infrastructure is quite centralized. What you're seeing with inference is in many use cases, as this becomes more ubiquitous, you're going to have more and more decentralized inference clusters.

Neil Tiwari: One thing I'm seeing is, for every application layer company out there, the highest line item from COGS is compute. And then the inference companies and inference clouds out there, most of them are purchasing up compute from either other clouds or unused capacity. And when you look at margins for that, you've got like layered margins. And so there's a push to kind of own your own infrastructure to really drive and increase profit margins, but also it's the ability to kind of have control of your own destiny. And I think a lot of folks are starting to, the application layer companies and inference clouds are grappling with how can we build and own and operate our own infrastructure.