Zeromatter

Further reading Robotics simulator (Wikipedia) · Physics engine (Wikipedia) · Lidar (Wikipedia) · Simulation City: Introducing Waymo's most advanced simulation system yet for autonomous driving (Waymo) · Co-simulation (Wikipedia)

1. i.

   Miles you can't drive

   Fatal crashes are rare compared with the miles people drive, which makes safety slow to prove. To show statistically that a self-driving car is safer than people, it would have to drive hundreds of millions of miles, sometimes hundreds of billions. Even under aggressive assumptions, existing fleets would need tens or hundreds of years to cover that. You cannot simply drive your way to safety.

2. ii.

   The tailgater who doesn't brake

   Pick a random tailgating scenario and the tailgater will probably brake in time. The interesting case is the one where they are distracted and don't. Real roads serve up those moments rarely and on their own schedule, so simulation is where developers go looking for them on purpose.

3. iii.

   The reality gap

   Real-world data is costly to gather and slow to accumulate, so simulation is attractive: it is a potentially infinite source of data and nobody gets hurt. The catch is that the gap between simulated and real worlds degrades a trained policyThe learned rule a robot uses to turn what it senses into what it does next. once it moves onto a real robot. A robot that aces the practice world can stumble on the first real floor.

4. iv.

   Speed against accuracy

   Robots that act on what their sensors tell them are much harder to simulate, because their motion depends on instant readings from the real world. Getting the physics very precise costs more processing power, which is why fast engines cut corners. Yet what happens in simulation has to predict what happens on the road, or the whole exercise is just a very expensive video game.

Further reading Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability? (RAND Corporation) · Simulation City: Introducing Waymo's most advanced simulation system yet for autonomous driving (Waymo) · Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (arXiv) · Robotics simulator (Wikipedia) · Physics engine (Wikipedia)

Further reading Join us (Zeromatter) · Company (Zeromatter) · One platform to build, test and train anything (Zeromatter)

1. Step 1: Sensors, faked well

   At the core is a cloud-based sensor simulation platform. The rendering work aims for virtual worlds that look real to the sensors that power autonomous systems, and to human eyes too. A separate sensor characterization effort tries to match each real sensor to its digital twinA software model built to behave like a specific real object, such as one particular sensor..

2. Step 2: Worlds built by recipe

   The platform generates environments automatically and runs multi-agent co-simulation, so many actors share a scene. Procedural tools, including Houdini, build varied worlds meant to challenge a robot's perception.

3. Step 3: Physics for every actor

   An in-house physics engine gives every simulated vehicle, aircraft, drone and robot physically accurate behaviour, with control systems on top. The range runs from cars in city traffic to an airliner taxiing out for take-off.

4. Step 4: Checking against hardware

   A robotics team trains policies in the simulator, then validates and iterates on real hardware, with the aim of closing the sim-to-real gapThe drop in performance when something trained in simulation meets the real world.. The company also helps customers certify autonomous systems by documenting how their algorithms were developed, trained and evaluated on its synthetic data.

Further reading Join us (Zeromatter) · One platform to build, test and train anything (Zeromatter)

• Does a simulator need to look real?

  One well-known experiment trained a real-world object detector, precise enough to guide a robot's grasp, on nothing but simulated images with deliberately unrealistic random textures. The idea is that with enough variety, the real world looks like one more variation. Photorealism and randomness pull in different directions, and both have their champions.

• When does a simulated mile count as evidence?

  Test driving alone can't prove safety, so developers need new ways to demonstrate it, and even then it may not be possible to establish safety with certainty. One sign of progress: as developers simulate more variations of a scenario, the spread of outcomes in simulation starts to match what they see on real roads.

Further reading Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World (arXiv) · Simulation City: Introducing Waymo's most advanced simulation system yet for autonomous driving (Waymo) · Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability? (RAND Corporation)

robotics simulator

Software that stands in for a physical robot and its surroundings so its programs can be written and tested on a computer.

physics engine

Software that approximates how objects move, collide and respond to forces.

lidar

A sensor that measures distance by timing how long a laser pulse takes to bounce back.

co-simulation

Running several separate simulations side by side that exchange data as they go.

policy

The learned rule a robot uses to turn what it senses into what it does next.

sim-to-real gap

The drop in performance when something trained in simulation meets the real world.

digital twin

A software model built to behave like a specific real object, such as one particular sensor.

1. 1Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability? · RAND Corporation
2. 2Simulation City: Introducing Waymo's most advanced simulation system yet for autonomous driving · Waymo
3. 3Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World · arXiv
4. 4Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey · arXiv
5. 5Robotics simulator · Wikipedia
6. 6Physics engine · Wikipedia
7. 7Lidar · Wikipedia
8. 8Co-simulation · Wikipedia
9. 9One platform to build, test and train anything · Zeromatter
10. 10Company · Zeromatter
11. 11Join us · Zeromatter