Video · 2026-07-02 · 15m · 30 moments

AI Factories Explained: What’s Actually New?

✦ AI generated

GUEST Thierry Pienaar GUEST Kaushik Shirhatti

timeline · colored by role

Definition◆

An AI factory, at any scale, is a system where you put in data and energy and, as if by magic, intelligence, tokens, and business outcomes come out — structured across five layers: energy, chips, infrastructure, models, and applications.

Kaushik frames the 'AI factory' concept as a universal five-layer model — energy, chips, infrastructure, models, applications — that converts data and energy into business outcomes regardless of scale.

transcript

Kaushik Shirhatti: at a basic fundamental level, what we think about an AI factory, doesn't matter if it's big or small, is when you put data in, you put energy in, and boom, like magic, intelligence, tokens, business outcomes come out. The way HPE and NVIDIA thinks about this AI factory is essentially across five layers.

Definition◆

An AI factory is a five-layer system — energy, chips, infrastructure, models, and applications — that turns data and energy input into intelligence, tokens, and business outcomes, regardless of scale.

Kaushik Shirhatti defines an 'AI factory' as a five-layer stack — energy, chips, infrastructure, models, applications — that converts data and energy into intelligence, tokens, and business outcomes.

transcript

Definition◆

An AI factory is fundamentally defined by feeding data and energy into a five-layer stack—energy, chips, infrastructure, models, and applications—to produce intelligence and business outcomes.

Kaushik Shirhatti lays out NVIDIA and HPE's shared framework for what an 'AI factory' actually is: a five-layer system spanning energy, chips, infrastructure, models, and applications that turns data and power into tokens and business outcomes.

transcript

Definition

An AI factory, regardless of scale, is a system that takes data and energy as inputs and produces intelligence, tokens, and business outcomes, structured across five layers: energy, chips, infrastructure, models, and applications.

Kaushik Shirhatti defines an AI factory as a data-and-energy-in, intelligence-and-outcomes-out system spanning five layers from energy to applications.

transcript

Kaushik Shirhatti: matter if it's big or small, is when you put data in, you put energy in, and boom, like magic, intelligence, tokens, business outcomes come out. The way HPE and NVIDIA thinks about this AI factory is essentially across five layers. So you think about the energy You think about the chips, so all the innovation that NVIDIA brings in, all the innovation that HPE brings in.

Definition◆

An AI factory, regardless of scale, is a system where you put data and energy in and get intelligence, tokens, and business outcomes out, organized across five layers: energy, chips, infrastructure, models, and applications.

Kaushik Shirhatti defines an AI factory not by size but as a five-layer system spanning energy, chips, infrastructure, models, and applications that turns data and power into business outcomes.

transcript

Kaushik Shirhatti: when you put data in, you put energy in, and boom, like magic, intelligence, tokens, business outcomes come out. The way HPE and NVIDIA thinks about this AI factory is essentially across five layers. So you think about the energy You think about the chips, so all the innovation that NVIDIA brings in, all the innovation that HPE brings in.

extends · 1

Context

AI factory adoption spans a spectrum from customers just starting with sixteen GPUs for generative AI to organizations with decades of HPC experience now transforming their HPC environments with AI.

Thierry Pienaar describes how HPE's AI factory concept flexes to fit customers ranging from small-scale generative AI beginners to sophisticated, decades-long HPC users.

transcript

Thierry Pienaar: we have customers that need sixteen GPUs, and they need to do generative AI, and they're just getting onto their journey. We have other customers that have been doing HPC for twenty, thirty years and understand traditional HPC environments. Now they're moving into AI and using AI to transform HPC, but they're highly sophisticated.

Context

AI factory adoption spans a wide spectrum, from small customers running 16 GPUs on early generative AI projects to highly sophisticated organizations with decades of HPC experience now transforming that infrastructure with AI.

Thierry Pienaar explains that the 'AI factory' concept flexes across a maturity spectrum, from newcomers with a handful of GPUs to seasoned HPC shops now applying AI to transform their existing environments.

transcript

Thierry Pienaar: If you think about the scale, the maturity of different customers, we have customers that need sixteen GPUs, and they need to do generative AI, and they're just getting onto their journey. We have other customers that have been doing HPC for twenty, thirty years and understand traditional HPC environments. Now they're moving into AI and using AI to transform HPC, but they're highly sophisticated.

Claim

AI factory needs range from enterprises just starting their generative AI journey with sixteen GPUs to highly sophisticated organizations with decades of HPC experience now using AI to transform HPC.

Thierry Pienaar explains that AI factory deployments span a maturity spectrum, from small generative-AI newcomers to long-time HPC shops now transforming their operations with AI.

transcript

Context

AI factory strategy has to flex with customer maturity — from newcomers who just need sixteen GPUs to run generative AI, to organizations with decades of HPC experience now transforming that infrastructure with AI.

Thierry explains that HPE sees AI factory needs spanning a spectrum, from small generative-AI starters to sophisticated, decades-long HPC shops now applying AI to transform their environments.

transcript

Mechanism

Efficient AI inference requires optimizing cost per token through memory technologies like KV cache and CXL, plus intelligent routing tools like Dynamo, matched to the right combination of GPU, CPU, and LPU.

Kaushik Shirhatti describes inference optimization as a combination of cost-per-token thinking and architectural strategy, using memory tech and routing tools to fit workloads to the right hardware.

transcript

Kaushik Shirhatti: You have to look at inference from a cost optimization perspective, power to token, and then also from a, let's say, architectural strategy perspective. So you can see the adoption of inference with technologies like memory optimizations, like KB Cache, CXL, intelligent routing capabilities like Dynamo from NVIDIA as well And so the ability to optimize, from the request all the way to the token, is load balancing cost fit for purpose for that workload.

explains mechanism · 2extends · 1gives example · 1

Mechanism

Optimizing inference requires a cost-per-token and architectural strategy, combining memory technologies like KV cache and CXL with intelligent routing tools like NVIDIA's Dynamo to balance workloads across GPU, CPU, or LPU types.

Kaushik Shirhatti explains that inference optimization is a cost-and-architecture problem solved with memory technologies and intelligent routing across mixed compute types.

transcript

extends · 1gives example · 1

Mechanism

Optimizing inference requires balancing cost-per-token against architecture choices, using memory technologies like KV cache and CXL plus intelligent routing like NVIDIA's Dynamo to match workloads to the right GPU, CPU, or LPU.

Thierry Pienaar explains that optimizing inference at scale means looking at cost-per-token and architecture together, leveraging memory technologies like KV cache and CXL alongside intelligent routing tools such as NVIDIA's Dynamo to match each workload to the right processor and memory strategy.

transcript

Thierry Pienaar: You have to look at inference from a cost optimization perspective, power to token, and then also from a, let's say, architectural strategy perspective. So you can see the adoption of inference with technologies like memory optimizations, like KB Cache, CXL, intelligent routing capabilities like Dynamo from NVIDIA as well.

extends · 1

Data

Enterprise AI ROI has jumped from around 5% a year ago to roughly 30% now, depending on the segment.

Thierry Pienaar cites a sharp rise in measured enterprise AI ROI, from about 5% a year ago to around 30% today in some segments.

transcript

Thierry Pienaar: a year ago, NVIDIA has done some independent studies and others as well, and we were at the five percent range for ROI. I think we're now moving up to the 30% range, depending on which segment you're looking at. But as a generality, we're seeing a, big uptick, I think, in, in true ROI return, implementing AI into the organizations.

rebuts · 1supports · 1

Data

Enterprise AI ROI has jumped roughly six-fold in about a year, from around 5% to as much as 30% depending on the segment.

Thierry Pienaar cites NVIDIA-backed research showing enterprise AI ROI climbing from around 5% a year ago to as high as 30% now, reflecting a broader shift from experimentation toward solving concrete problem statements.

transcript

Thierry Pienaar: a year ago, NVIDIA has done some independent studies and others as well, and we were at the five percent range for ROI. I think we're now moving up to the 30% range, depending on which segment you're looking at. But as a generality, we're seeing a big uptick in true ROI return, implementing AI into the organizations.

Data

Enterprise AI ROI has risen sharply, from around 5% a year ago to around 30% now, even as data challenges like locating, curating, and protecting data remain significant.

Thierry Pienaar cites NVIDIA-linked research showing enterprise AI ROI jumping from about 5% to about 30% over the past year, alongside ongoing data-management challenges.

transcript

rebuts · 1

Data◆

Enterprise ROI from AI implementation has jumped from roughly 5% a year ago to around 30% today, depending on the segment.

Citing NVIDIA and independent studies, Thierry says measured enterprise ROI from AI has surged from about 5% a year ago to roughly 30% now.

transcript

Thierry Pienaar: We're seeing a, an uplift, I think, in ROI, right? So I think a year ago, NVIDIA has done some independent studies and others as well, and we were at the five percent range for ROI. I think we're now moving up to the 30% range, depending on which segment you're looking at.

supports · 1

Data

Enterprise AI ROI has climbed sharply, from roughly 5% a year ago to roughly 30% today depending on segment, as organizations move past experimentation toward solving concrete problem statements.

Kaushik Shirhatti cites NVIDIA-linked research showing enterprise AI ROI jumping from about 5% a year ago to around 30% now, as companies shift from experimentation to targeted problem-solving.

transcript

Kaushik Shirhatti: a year ago, NVIDIA has done some independent studies and others as well, and we were at the five percent range for ROI. I think we're now moving up to the 30% range, depending on which segment you're looking at.

explains mechanism · 1

Example◆

Successful enterprise AI adoption comes from deliberately prioritizing a handful of projects to pursue deeply end-to-end, rather than chasing every idea a room full of people can generate.

Thierry Pienaar argues that intentional prioritization, picking a few projects and going deep on measurable workflow impact, is what separates successful AI adopters from those that merely generate ideas.

transcript

Thierry Pienaar: The places where I've seen people be very, successful is prioritization, right? And that continues to be a challenge, because, if you take a room full of 100 people They will, come up with 100 different amazing ideas how AI can transform a bank or a hospital. But really being intentional and saying, I'm gonna pick four or five projects, and we are going to go deep, and actually go see it end to end.

provides context · 1

Claim

The organizations that succeed with AI resist chasing every idea a room of people can generate and instead intentionally pick a handful of projects to pursue deeply, measuring the full workflow end to end.

Kaushik Shirhatti argues the most successful companies avoid spreading effort across every possible AI idea and instead commit deeply to a small number of prioritized projects, measuring real end-to-end impact.

transcript

Kaushik Shirhatti: if you take a room full of 100 people They will, come up with 100 different amazing ideas how AI can transform a bank or a hospital. But really being intentional and saying, 'I'm gonna pick four or five projects, and we are going to go deep,' and actually go see it end to end, like the workflow, measure the productivity, how is it actually being useful.

extends · 1provides context · 1

Claim◆

Successful AI adoption comes from intentionally prioritizing a handful of projects and pursuing them deeply, end-to-end, rather than spreading effort thin across many ideas.

Kaushik Shirhatti argues that the companies most successful with AI are the ones that deliberately narrow focus to a few projects and see them through end-to-end.

transcript

Kaushik Shirhatti: really being intentional and saying, 'I'm gonna pick four or five projects, and we are going to go deep,' and actually go see it end to end, like the workflow, measure the productivity, how is it actually being useful.

extends · 1

Claim

Companies succeed with AI when they move beyond optimizing existing processes to fully reimagining them, which makes the ROI obvious and drives organization-wide adoption.

Thierry Pienaar argues that the biggest driver of successful AI adoption isn't incremental process optimization but a willingness to fully reimagine workflows end-to-end—once done well, the ROI becomes obvious and adoption spreads rapidly.

transcript

Thierry Pienaar: there's a very subtle difference between optimizing a process and reimagining a process. Once you actually put an effort into reimagining a process and you've drawn the workflows, the ROI, and you have-- you've done-- assuming you've done it well, the ROI is just so obvious, and then the adoption just goes... skyrocket... across the organization.

explains mechanism · 1supports · 2

Claim◆

A strong sovereign AI strategy is one of the most powerful levers a country has to drive economic value, because it lets nations use their own language, culture, and context without depending on foreign infrastructure.

Kaushik Shirhatti argues that sovereign AI has become critical because it lets nations build economic value using their own language and cultural context while reducing dependence on foreign infrastructure, citing examples in the Middle East, France, India, and Japan.

transcript

Claim◆

Sovereign AI is critical for nations because a strong sovereign AI strategy is the biggest lever they have to drive economic value for their country and citizens, using their own language and cultural context instead of depending on another country's infrastructure.

Kaushik Shirhatti argues sovereign AI has become critical for nations, since it lets them capture economic value, preserve their language and cultural context, and avoid dependence on infrastructure controlled by other countries.

transcript

Kaushik Shirhatti: Sovereign AI, to me, is just so critical for any country, right? Is because most of these countries who are adopting sovereign AI strategy aggressively have realized one thing, which is a good sovereign AI strategy is the biggest place where they can drive economic value for the country, for the citizens. They have the language, they have the context, they have the cultural context.

Claim◆

Sovereign AI is critical for nations because it lets them capture economic value while preserving their own language and cultural context instead of depending on foreign infrastructure.

Kaushik Shirhatti says sovereign AI has become essential for countries seeking economic value and independence, driven by control over language, context, and infrastructure.

transcript

Kaushik Shirhatti: Sovereign AI, to me, is just so critical for any country, right? Is because most of these countries who are adopting sov-sovereign AI strategy aggressively have realized one thing, which is a good sovereign AI strategy is the biggest Place where they can drive economic value for the country, for the citizens. They have the language, they have the context, they have the cultural context.

extends · 1gives example · 1supports · 1

Claim

Sovereign AI strategy is critical for any country because it drives economic value for citizens while reducing dependence on foreign infrastructure, and preserving local language and cultural context.

Kaushik Shirhatti says nations pursuing sovereign AI are recognizing it as their biggest lever for economic value and independence, preserving language and cultural context instead of relying on foreign infrastructure.

transcript

extends · 1supports · 1

Claim◆

Sovereign AI is fundamentally about strategic autonomy for a nation or enterprise — minimizing dependency on outside infrastructure — with TELUS as a prime example, building talent, infrastructure, and software ecosystems to serve Canadian citizen services, defense, and economic empowerment.

Kaushik describes sovereign AI as strategic autonomy — never eliminating all dependency, but reducing it — pointing to TELUS as a Canadian telco example spanning citizen services, defense simulation, and startup ecosystem support.

transcript

Kaushik Shirhatti: Sovereign AI, at the end of the day, really is about strategic autonomy, right? It is about a nation or an entity, it can be at the enterprise level as well, right? really not being dependent, as little dependency as possible... TELUS is a great example of that because they're fundamentally a Canadian telco that is servicing in the interests of Canada.

gives example · 1

Prediction

Within a couple of years, liquid cooling will be standard across the board and agentic AI capabilities will be as ubiquitous and unremarkable in daily work as using Excel or PowerPoint.

Thierry Pienaar predicts that in a few years liquid cooling will be the norm across data centers and agentic AI will be as routine a daily tool for end users as Excel or PowerPoint.

transcript

Thierry Pienaar: I think we'll definitely be at liquid cooling across the board. I think that's something that will happen, and I think that, agentic capabilities will be every day. I think it'll be like using Excel, using PowerPoint, I think, and it's end users that will be empowered as well to do that.

Prediction

Within a few years, liquid cooling will be standard across the industry and agentic AI will be as ordinary and everyday a tool as Excel or PowerPoint, with its complexity abstracted away from end users.

Thierry predicts liquid cooling becomes ubiquitous and agentic AI turns into a mundane, everyday tool for end users, as commonplace as Excel or PowerPoint.

transcript

Prediction◆

The on-prem-versus-cloud debate will soon look as outdated as asking whether you need email servers, since customers will simply choose hybrid setups per workload, and 'AI factory' budgeting will stop being a distinct question — while physical AI/robotics looms as the next big leap after agentic AI.

Kaushik predicts the on-prem-vs-cloud framing will disappear as hybrid becomes default and 'AI factory' stops being a special budget line, with physical AI (robots with agentic-level capability) as the next frontier.

transcript

Kaushik Shirhatti: I think this whole on-prem versus cloud debate, I think, will be like, 'What were we thinking?' I think customers are gonna choose hybrid. They are gonna smart enough to pick what workloads go on cloud, which one will go on prem... just like we don't ask whether we need email servers, we're not gonna ask, do we need AI factories?

Prediction

Physical AI—robots with the full power and capability of agentic AI—is coming soon and will make today's excitement about agentic AI look as quaint as early excitement about gen AI now seems.

Kaushik Shirhatti predicts that physical AI—robots endowed with full agentic AI capability—is imminent, and that today's excitement over agentic AI will soon look as quaint as early enthusiasm for generative AI does now.

transcript

Kaushik Shirhatti: just like how we feel about gen AI versus agentic AI now. I think physical AI is gonna be around the corner, and physical AI, when these machines and robots have all the power and capability of like agentic AI, we're gonna look back at this time and say, 'Oh, is that, remember the time when we were like all excited about agentic AI?'

Highlight slides

The AI Factory: A Universal Model✦ from: An AI factory, at any scale, is a system where you put in data and energy and, as if by magic, intelligence, tokens, and business outcomes come out — structured across five layers: energy, chips, infrastructure, models, and applications.Five Layers of an AI Factory✦ from: An AI factory, at any scale, is a system where you put in data and energy and, as if by magic, intelligence, tokens, and business outcomes come out — structured across five layers: energy, chips, infrastructure, models, and applications.What Is an AI Factory?✦ from: An AI factory is fundamentally defined by feeding data and energy into a five-layer stack—energy, chips, infrastructure, models, and applications—to produce intelligence and business outcomes.What Is an AI Factory?✦ from: An AI factory is a five-layer system — energy, chips, infrastructure, models, and applications — that turns data and energy input into intelligence, tokens, and business outcomes, regardless of scale.What Is an AI Factory?✦ from: An AI factory, regardless of scale, is a system where you put data and energy in and get intelligence, tokens, and business outcomes out, organized across five layers: energy, chips, infrastructure, models, and applications.Five Layers of an AI Factory✦ from: An AI factory, regardless of scale, is a system where you put data and energy in and get intelligence, tokens, and business outcomes out, organized across five layers: energy, chips, infrastructure, models, and applications.The Five-Layer Stack✦ from: An AI factory is a five-layer system — energy, chips, infrastructure, models, and applications — that turns data and energy input into intelligence, tokens, and business outcomes, regardless of scale.The Five-Layer Stack✦ from: An AI factory is fundamentally defined by feeding data and energy into a five-layer stack—energy, chips, infrastructure, models, and applications—to produce intelligence and business outcomes.From Layers to Outcomes✦ from: An AI factory is fundamentally defined by feeding data and energy into a five-layer stack—energy, chips, infrastructure, models, and applications—to produce intelligence and business outcomes.Enterprise AI ROI Surges✦ from: Enterprise ROI from AI implementation has jumped from roughly 5% a year ago to around 30% today, depending on the segment.Source of the Data✦ from: Enterprise ROI from AI implementation has jumped from roughly 5% a year ago to around 30% today, depending on the segment.Prioritization Beats Ideation✦ from: Successful enterprise AI adoption comes from deliberately prioritizing a handful of projects to pursue deeply end-to-end, rather than chasing every idea a room full of people can generate.Go Deep, Not Wide✦ from: Successful enterprise AI adoption comes from deliberately prioritizing a handful of projects to pursue deeply end-to-end, rather than chasing every idea a room full of people can generate.Focus Beats Breadth in AI Adoption✦ from: Successful AI adoption comes from intentionally prioritizing a handful of projects and pursuing them deeply, end-to-end, rather than spreading effort thin across many ideas.What 'Going Deep' Looks Like✦ from: Successful AI adoption comes from intentionally prioritizing a handful of projects and pursuing them deeply, end-to-end, rather than spreading effort thin across many ideas.Why Sovereign AI Is Critical✦ from: Sovereign AI is critical for nations because a strong sovereign AI strategy is the biggest lever they have to drive economic value for their country and citizens, using their own language and cultural context instead of depending on another country's infrastructure.Why Sovereign AI Matters✦ from: Sovereign AI is critical for nations because it lets them capture economic value while preserving their own language and cultural context instead of depending on foreign infrastructure.Sovereign AI: A Lever for Economic Value✦ from: A strong sovereign AI strategy is one of the most powerful levers a country has to drive economic value, because it lets nations use their own language, culture, and context without depending on foreign infrastructure.Core Drivers✦ from: Sovereign AI is critical for nations because it lets them capture economic value while preserving their own language and cultural context instead of depending on foreign infrastructure.Core Drivers of Sovereign AI Strategy✦ from: Sovereign AI is critical for nations because a strong sovereign AI strategy is the biggest lever they have to drive economic value for their country and citizens, using their own language and cultural context instead of depending on another country's infrastructure.Where Sovereign AI Is Taking Hold✦ from: A strong sovereign AI strategy is one of the most powerful levers a country has to drive economic value, because it lets nations use their own language, culture, and context without depending on foreign infrastructure.Global Momentum✦ from: A strong sovereign AI strategy is one of the most powerful levers a country has to drive economic value, because it lets nations use their own language, culture, and context without depending on foreign infrastructure.Sovereign AI = Strategic Autonomy✦ from: Sovereign AI is fundamentally about strategic autonomy for a nation or enterprise — minimizing dependency on outside infrastructure — with TELUS as a prime example, building talent, infrastructure, and software ecosystems to serve Canadian citizen services, defense, and economic empowerment.TELUS: Canadian Sovereign AI in Action✦ from: Sovereign AI is fundamentally about strategic autonomy for a nation or enterprise — minimizing dependency on outside infrastructure — with TELUS as a prime example, building talent, infrastructure, and software ecosystems to serve Canadian citizen services, defense, and economic empowerment.On-Prem vs. Cloud Debate Will Fade✦ from: The on-prem-versus-cloud debate will soon look as outdated as asking whether you need email servers, since customers will simply choose hybrid setups per workload, and 'AI factory' budgeting will stop being a distinct question — while physical AI/robotics looms as the next big leap after agentic AI.Next Frontier: Physical AI✦ from: The on-prem-versus-cloud debate will soon look as outdated as asking whether you need email servers, since customers will simply choose hybrid setups per workload, and 'AI factory' budgeting will stop being a distinct question — while physical AI/robotics looms as the next big leap after agentic AI.

Related episodes

Video · The Cognitive Revolution

Fable's Back, AI Engineer Recap, & SambaNova

→

Video · 20VC

OpenRouter CEO: Why Chinese Open Models Are Beating the US | Why Enterprises Fear OpenAI & Anthropic

→

Audio · All-In Podcast

OpenAI CFO Sarah Friar: IPO, AI Rivalries, New Device, and Spending $100B+ on Compute

→