The key role simulation plays that real-world data cannot is counterfactual reasoning—playing out events that haven't happened or cannot happen—which is essential because robotics cannot possibly collect enough real-world data for all scenarios.
Fei-Fei Li argues simulation is not a binary alternative to real data but serves a unique role: counterfactual reasoning. She cites Waymo's billions of hours of simulation as a real-world example. ✦ AI generated
Fei-Fei Li · a16z Podcast · 2026-07-28 · original ↗
starts at this moment · 19:47
“As an investor, I've heard other researchers say like Sergey Levine say simulation will always eventually deviate from the physical world and real world data collection is absolutely critical.”
There isn't a binary choice between simulation or no simulation. All this come together to make robotics work. Think about human intelligence. We do a lot of simulation in our head. Why? There's a very important role simulation plays that real world data doesn't play which is counterfactual reasoning—you play out events that hasn't happened or cannot happen or you don't have enough data to make it happen in real world. And while you play it out, you learn how to act in it. Humans do this all the time. [...] Here's a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation and actually Waymo is more simulation-heavy than just real world data heavy. So these are real examples and as you know cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning.
verbatim transcript · starts at 19:47
19:47binary choice between simulation or no simulation. All this come um in together um to to make robotics work. Think about human intelligence. We do a lot of simulation in our head. You know why? There's a very important role simulation plays that real world data doesn't play which is counterfactual reasoning is that you play out events that c hasn't happened or cannot happen or you don't have enough data to make it happen in
20:18real world. And while you play it out, you learn how to act in it. Humans do this all the time. We probably don't, you know, we just I know you were at World Cups. >> I was at the World. >> Congratulations to Spain winning. I'm sure in the planning of every game there is simulation, whether it's digital or on the on the whiteboard or whatever, that simulation, the role simulation
20:44plays is counterfactual reasoning. And that's really important in robotics because we just do not have cannot possibly have enough real world data for that. Here's a real life uh example the the industry of self-driving cars. Whimo has officially said they use billions of hours of simulation >> and and actually Whimo is more simulationheavy than just real world data heavy. So these are real examples and and as you know Martin Andrew too
21:17cars are the simplest kind of robots. >> Yeah. >> Yeah. So so clearly simulation plays a huge role in robotic learning. >> I also want to add to that. So like there are if you put things more specific simulation can provide two levels of benefits. The first one is reliability and the second one is efficiency. So for reliability, if you're thinking about a robotic system working reliable in the real
21:43environments, you need data to provide systematic coverage of all the state space and the variations that robots might encounter. That's how you can learn of how that is robust. So with simulation, you can do systematic randomizations and control and the variations of lighting, frictions, geometries, object types and also all different kind of physical parameters to making sure you have sufficient coverage of the state space. So this is what can