ContextVideo · 19:59 — 21:26

Simulation is not an alternative to real-world data but plays a critical complementary role that real data cannot fill: counterfactual reasoning — playing out events that haven't happened, cannot happen, or lack sufficient real-world data.

Fei-Fei Li argues that simulation and real-world data are not a binary choice — simulation enables counterfactual reasoning essential for robotics, citing Waymo's billions of simulation hours and human cognitive simulation as examples. ✦ AI generated

Fei-Fei Li · a16z Podcast · 2026-07-28 · original ↗

Share card ↓Export .md Export .txt

starts at this moment · 19:59

There isn't a binary choice between simulation or no simulation. All this come together to make robotics work. Think about human intelligence. We do a lot of simulation in our head. Why? There's a very important role simulation plays that real world data doesn't play which is counterfactual reasoning — you play out events that hasn't happened or cannot happen or you don't have enough data to make it happen in real world. And while you play it out, you learn how to act in it. Humans do this all the time. Here's a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation and actually Waymo is more simulation-heavy than just real world data heavy. So clearly simulation plays a huge role in robotic learning.

verbatim transcript · starts at 19:59

Transcript · around this moment

19:59simulation in our head. You know why? There's a very important role simulation plays that real world data doesn't play which is counterfactual reasoning is that you play out events that c hasn't happened or cannot happen or you don't have enough data to make it happen in real world. And while you play it out, you learn how to act in it. Humans do this all the time. We probably don't,

20:27you know, we just I know you were at World Cups. >> I was at the World. >> Congratulations to Spain winning. I'm sure in the planning of every game there is simulation, whether it's digital or on the on the whiteboard or whatever, that simulation, the role simulation plays is counterfactual reasoning. And that's really important in robotics because we just do not have cannot possibly have enough real world data for

20:54that. Here's a real life uh example the the industry of self-driving cars. Whimo has officially said they use billions of hours of simulation >> and and actually Whimo is more simulationheavy than just real world data heavy. So these are real examples and and as you know Martin Andrew too cars are the simplest kind of robots. >> Yeah. >> Yeah. So so clearly simulation plays a huge role in robotic learning.

21:26>> I also want to add to that. So like there are if you put things more specific simulation can provide two levels of benefits. The first one is reliability and the second one is efficiency. So for reliability, if you're thinking about a robotic system working reliable in the real environments, you need data to provide systematic coverage of all the state space and the variations that robots might encounter. That's how you can

21:52learn of how that is robust. So with simulation, you can do systematic randomizations and control and the variations of lighting, frictions, geometries, object types and also all different kind of physical parameters to making sure you have sufficient coverage of the state space. So this is what can give the robotic systems reliability and second is about efficiency. So right now many people are doing tele operation and if you look at many of the television

▦ Waymo ◉ Fei-Fei Li

Around this claim

Mechanism · 1

Simulation plays a critical role in robotic learning that real-world data cannot replace — it enables counterfactual reasoning, which is essential for robotics.Fei-Fei Li · a16z Podcast · conf 95%

Evidence · 3

The key role simulation plays that real-world data cannot is counterfactual reasoning—playing out events that haven't happened or cannot happen—which is essential because robotics cannot possibly collect enough real-world data for all scenarios.Fei-Fei Li · a16z Podcast · conf 100%Simulation plays an essential role in robotics that real-world data cannot replace because it enables counterfactual reasoning — playing out events that haven't happened or cannot happen — and real-world deployments like Waymo already prove its value.Fei-Fei Li · a16z Podcast · conf 95%The real market need is not prediction of the future but causal counterfactuals — understanding how to shape the future.Jun Park · 20VC · conf 90%

Extends · 2

Simulation provides two unique benefits that real-world data cannot: systematic coverage for reliability (controllable variation of all state-space parameters) and accelerated data generation for efficiency (faster-than-real-time training).Yunzhu Li · a16z Podcast · conf 85%The real-to-sim-to-real pipeline maps real environments into aligned digital worlds to replace costly real-world data collection with scalable simulation data for robotics training and evaluation.Yunzhu Li · a16z Podcast · conf 70%

This moment responds to

supports → The role simulation plays that real-world data cannot is counterfactual reasoning—playing out events that haven't happened or cannot happen—and this is critical for robotics because we cannot possibly have enough real-world data.Fei-Fei Li · a16z Podcast explains mechanism → Simile creates a foundation model of human behavior — not of rationality — that is designed to make the same mistakes and biases humans do, representing the subjective half of the brain.Jun Park · 20VC explains mechanism → The next frontier of AI is spatial intelligence — creating AI that can generate, understand, reason with, and interact with physical or virtual spaces.Fei-Fei Li · a16z Podcast extends → Simulation and real-world data collection are not contradictory; they operate in a data flywheel where physics-based simulation provides initial structure and consistency for robot training, then real-world data accumulation shifts toward more learning-based modeling over time.Yunu Guo · a16z Podcast