Simulation plays an essential role in robotics that real-world data cannot replace because it enables counterfactual reasoning — playing out events that haven't happened or cannot happen — and real-world deployments like Waymo already prove its value.
Fei-Fei Li argues there is no binary choice between simulation and real-world data: simulation is essential for counterfactual reasoning, and Waymo's billions of simulation hours prove it works at scale. ✦ AI generated
Fei-Fei Li · a16z Podcast · 2026-07-28 · original ↗
starts at this moment · 19:47
“As an investor, I've heard other researchers say like Sergey Levine say simulation will always eventually deviate from the physical world and real world data collection is absolutely critical. And so maybe talk a little bit about the viability of this approach where simulation is a cornerstone as opposed to some other approach.”
I want to add to this and be slightly philosophical here is there isn't a a a binary choice between simulation or no simulation. All this come um in together um to to make robotics work. Think about human intelligence. We do a lot of simulation in our head. You know why? There's a very important role simulation plays that real world data doesn't play which is counterfactual reasoning is that you play out events that c hasn't happened or cannot happen or you don't have enough data to make it happen in real world. And while you play it out, you learn how to act in it. Humans do this all the time. We probably don't, you know, we just I know you were at World Cups. I was at the World. Congratulations to Spain winning. I'm sure in the planning of every game there is simulation, whether it's digital or on the on the whiteboard or whatever, that simulation, the role simulation plays is counterfactual reasoning. And that's really important in robotics because we just do not have cannot possibly have enough real world data for that. Here's a real life uh example the the industry of self-driving cars. Whimo has officially said they use billions of hours of simulation and and actually Whimo is more simulationheavy than just real world data heavy. So these are real examples and and as you know Martin Andrew too cars are the simplest kind of robots. Yeah. So so clearly simulation plays a huge role in robotic learning.
verbatim transcript · starts at 19:47
19:47binary choice between simulation or no simulation. All this come um in together um to to make robotics work. Think about human intelligence. We do a lot of simulation in our head. You know why? There's a very important role simulation plays that real world data doesn't play which is counterfactual reasoning is that you play out events that c hasn't happened or cannot happen or you don't have enough data to make it happen in
20:18real world. And while you play it out, you learn how to act in it. Humans do this all the time. We probably don't, you know, we just I know you were at World Cups. >> I was at the World. >> Congratulations to Spain winning. I'm sure in the planning of every game there is simulation, whether it's digital or on the on the whiteboard or whatever, that simulation, the role simulation
20:44plays is counterfactual reasoning. And that's really important in robotics because we just do not have cannot possibly have enough real world data for that. Here's a real life uh example the the industry of self-driving cars. Whimo has officially said they use billions of hours of simulation >> and and actually Whimo is more simulationheavy than just real world data heavy. So these are real examples and and as you know Martin Andrew too
21:17cars are the simplest kind of robots. >> Yeah. >> Yeah. So so clearly simulation plays a huge role in robotic learning. >> I also want to add to that. So like there are if you put things more specific simulation can provide two levels of benefits. The first one is reliability and the second one is efficiency. So for reliability, if you're thinking about a robotic system working reliable in the real
21:43environments, you need data to provide systematic coverage of all the state space and the variations that robots might encounter. That's how you can learn of how that is robust. So with simulation, you can do systematic randomizations and control and the variations of lighting, frictions, geometries, object types and also all different kind of physical parameters to making sure you have sufficient coverage of the state space. So this is what can
- ·Simulation vs real-world data is a false binary
- ·Both work together to make robotics succeed
- ·Simulation plays a role real data cannot replace
- ·Play out events that haven't happened or cannot happen
- ·Learn how to act without requiring real-world data
- ·Humans do this instinctively (e.g. pre-game planning)
- ·Waymo uses billions of hours of simulation
- ·Simulation-heavy approach, not just data-heavy
- ·Self-driving cars are proof simulation works in deployment