In California, Which Self-Driving Cars Log the Most Miles?

How many times in 2019 did human safety operators have to take control of self-driving vehicles being tested in California? With the release of new reports Wednesday by the state's Department of Motor Vehicles, we can begin to know. What's more, robot cars in autonomous mode drove almost 2.9 million miles on public California roads last year, up from 2.1 million miles in 2018.

For various reasons, disengagements are a crummy tool for measuring progress toward safe, efficient autonomy, and for comparing momentum across companies. For one, the players don’t always define “disengagement” in the same way. Plus, lots of the big autonomous vehicle players—Waymo, Uber, and Argo AI, among others—do the bulk of their testing outside of California. Companies have more sophisticated internal metrics for measuring their own success—you can read about those here. The reports also do not include driving on private test facilities and campuses, nor the many more “virtual miles” that autonomous vehicle systems travel in digital space.

Want the latest news on self-driving cars in your inbox? Sign up here!

But the reports aren’t devoid of interesting information. They include how many miles each company covered and how many cars it operates, and the reason for each disengagement. Taken together and compared with data from years past, those data reveal a few noteworthy trends and details about the ongoing race to develop the self-driving car.

Of the 36 companies that reported testing in California between December 1, 2018 and November 30, 2019, Cruise’s filing may reveal the most about its progress. That’s because the General Motors subsidiary does nearly all its public road testing in San Francisco, where it’s aiming to launch a robotaxi service in the next few years. CEO Dan Ammann last year delayed Cruise’s initial plan to launch in 2019, and said the company was building out its fleet of test vehicles to log more miles. Now, we know what he meant: Cruise ran 226 vehicles in California in 2019, up from 125 in 2018. It logged 831,000 miles, up from 447,600.

In the same stretch, Apple’s secretive self-driving effort built up its fleet, but effectively stopped driving on public roads. All of its disengagements occurred between June and November, indicating that its cars drove very little in the first half of the year. The company laid off more than 200 of its Project Titan employees in January 2019, then acquired (and hired employees from) the self-driving startup Drive.ai in June.

Among other players, startup Zoox nearly doubled its mileage compared with 2018, as it seeks to test its custom-built vehicle this year, ahead of a planned robotaxi business, the Verge reports. Waymo’s impressive 1,454,100 miles last year—the equivalent of almost 2,000 round trips between San Francisco and Los Angeles—represented a 15 percent increase from 2018. Nuro, which makes low-speed vehicles for food delivery, also piled on the distance. Its purpose-built vehicles are running pilots in Texas and Arizona; in California, Nuro tests with conventional cars retrofitted with self-driving smarts, with safety operators inside.

All told, safety operators across California took control from their robots about 9,000 times, but a few companies accounted for most of those: Lyft, Mercedes-Benz, and Toyota each reported more than 1,600 disengagements. State law requires carmakers to explain each disengagement. The result is a sea of vagaries, euphemisms, and jargon, including “Planning Module Failure due to software issue;” “a software fault due to a potential performance issue with a software component of the self-driving system (including third-party software components);” and “incorrectly perceived trajectory of a nearby object led to a planned trajectory that overlaps with a nearby cyclist.” But in the vastness lie a few interesting oases that help explain why these companies have spent tens of billions of dollars and more than a decade on this technology, and remain far from done.

Cruise blames 43 of its 68 disengagements at least in part on other people “behaving poorly.” That may refer to things that are technically illegal but socially acceptable, or at least predictable, such as a jaywalker or a quick three-point turn. Similarly (if more charitably), Zoox blames many of its disengagements on its system’s failure to accurately predict where cyclists, pedestrians, scooters, and other vehicles are headed. Nuro drivers had to take over because their robot had trouble with palm fronds, garbage bags, and animals in the road. To recast Melisandre, the roads may be well lit, but are full of terrors. That means next year’s batch of disengagement reports may show progress, but don’t expect a finished product.