A summary of the bad software decisions made by Uber’s self driving car that lead to that fatal crash with a jay-walking pedestrian in 2018.
Like most self-driving software, Uber’s software tries to classify each object it detects into one of several categories—like car, bicycle, or “other.” Then, based on this classification, the software computes a speed and likely trajectory for the object. This system failed catastrophically in Tempe.
The NTSB report includes a second-by-second timeline showing what the software was “thinking” as it approached Herzberg, who was pushing a bicycle across a multi-lane road far from any crosswalk:
- 5.2 seconds before impact, the system classified her as an “other” object.
- 4.2 seconds before impact, she was reclassified as a vehicle.
- Between 3.8 and 2.7 seconds before impact, the classification alternated several times between “vehicle” and “other.”
- 2.6 seconds before impact, the system classified Herzberg and her bike as a bicycle.
- 1.5 seconds before impact she became “unknown.”
- 1.2 seconds before impact she became a “bicycle” again.
Two things are noteworthy about this sequence of events. First, at no point did the system classify her as a pedestrian. According to the NTSB, that’s because “the system design did not include consideration for jaywalking pedestrians.”
Second, the constantly switching classifications prevented Uber’s software from accurately computing her trajectory and realizing she was on a collision course with the vehicle. You might think that if a self-driving system sees an object moving into the path of the vehicle, it would put on its brakes even if it wasn’t sure what kind of object it was. But that’s not how Uber’s software worked.