Data Shows Promise for Autonomous Vehicles' Progress

The future is bright with regard to autonomous vehicles (AVs). In 2016 we saw the crest of the wave, but 2017 just might be the year self-driving cars become part of the cultural Zeitgeist. With that being said, many still hold reservations about the technology and are hesitant to forfeit the steering wheel and gas/brake pedals.

And it's because of this hesitation that certain AV manufacturers must share certain driving data with the state of California and its citizens — data released Feb. 1 by the California DMV that, when compared with 2015 figures, shows considerable promise in the technology's progression.

Disengagement Data Breakdown

In attempting to walk the tightrope between allowing innovation and crafting regulations to ensure safety and transparency, the California DMV released its draft of rules for testing AVs on the state's public roads in December 2015. The DMV invited industry input on many of the provisions, including the requirements that a human driver be present and capable of immediately taking over if necessary; manufacturers must apply for and be granted a testing permit; and driving data must be shared openly with the state.

Further still, part of the reported data — all disengagement data from when autonomous vehicles are being tested on public roads — also must be shared with the public. Disengagements are defined by the DMV (PDF) as deactivations of the autonomous mode in two situations:

1. When a failure of the autonomous technology is detected; or
2. When the safe operation of the vehicle requires that the autonomous vehicle test driver disengage the autonomous mode and take immediate manual control of the vehicle.

“The disengagement reports are a tool for the department to review to see if manufacturers are improving the technology over time,” DMV spokesperson Jessica Gonzalez told Government Technology.

Although 21 companies have been granted autonomous vehicle testing permits, only 11 manufacturers were required to submit a disengagement report this year because of the time in which the permits were granted:

	Miles Driven	Number of Disengagements
BMW	638	1
Bosch	983	1,442
GM Cruise	9,623	262
Delphi Automotive Systems	3,125	277
Ford	590	3
Google Auto/Waymo	635,868	124
Honda	N/A	N/A
Nissan	4,099	28
Mercedes-Benz	673	336
Tesla Motors	550	182
Volkswagen	N/A	N/A

It should be noted that this set of data only includes numbers derived from tests on California public roads. Honda, who has had an AV testing permit did not use public roads to test, and instead used the GoMentum Station in Contra Costa County. Likewise, much of the testing for GM and Ford is done in Michigan.

The data should not be seen as indicative of shortcomings in the technology itself. Often AV disengagements were due to human taking over controls because they became nervous and felt uneasy.

Disengagement Data Assessment

When assessing the data as a whole, it is hard to understate how impressive the technology has become: Self-driving cars, on average, drove nearly 250 miles without experiencing a disengagement. In the data frp, the 2015 reports, vehicles traveled an average of 183 miles before experiencing a disengagement. Though not all of the reports submitted for 2016 had counterparts for 2015 data, the increase in miles traveled is still significant.

Each report required detailed information about the disengagements including: testing conditions; what type of road the vehicle was on (interstate, highway, rural, street or parking facility); whether the disengagement was planned; the total number of miles driven by each car; and the reaction time of the autonomous vehicle test driver. Some companies even went further in providing exact times the disengagement occurred. GM's report on its Cruise vehicle included the car’s animal- and comic-book-inspired nicknames.

Some of the most interesting results come from looking at the data on a case-by-case basis.

2016-miles-per-disengagement.PNG

Google’s self-driving project, recently renamed Waymo, has logged 600,000 miles more than the next closest manufacturer, GM. The company was proud to point out that disengagements had dropped by 75 percent from 2015. Per 100 miles, the company experienced only 0.2 disengagements, while in 2015, the cars experienced 0.8 disengagements per 100 miles.

“We’ve been able to make dramatic improvements to our technology because we use each of these disengages to teach and refine our car,” said Dmitri Dolgov, head of Waymo’s self-driving technology, in a Medium post.

To fully appreciate how this technology has progressed, it is worth looking back at last year's reports on disengagement and compare them with 2016 statistics.

miles-per-disengagement-comparison.PNG

Another way to look at the differences between 2015 and 2016 is to see how many more (or fewer) miles autonomous vehicles from a few of the key manufacturers traveled without being disengaged from AV mode:

difference-in-disengagements-per-mile.PNG

Although it is difficult to predict how long it will take for full integration of autonomous vehicles onto public roadways, the progress is promising.