While Elon Musk was busy tweeting seemingly every update to Tesla’s Autopilot, the project that would become Waymo was quietly making huge strides in progress toward a self-driving car system. There are 11 companies developing autonomous cars in California, but new state reports of disengagements (where a human driver has to take over from an autonomous car) show that Waymo is out in front, traveling more miles under autonomous control than any of the other companies and reporting a far lower disengagement rate.
Of the 11 companies testing autonomous cars in California, Waymo is raking in the road miles over the last year and its disengagement rate is significantly lower than any of its competitors. Tesla has a remarkably high disengagement rate for the number of miles it drove, and is getting left behind by most of the other companies that reported to the California DMV.
Although disengagements aren’t the only way to measure autonomous cars, Tesla’s Autopilot certainly seems to be behind on this metric.
John Krafcik, CEO of Waymo, and a Waymo autonomous minivan.
California requires companies to report on the number of times autonomous cars have to switch back to being driver controlled, for failure or technical issues. Waymo had 124 disengagements over 635,868 miles of autonomous road testing. Tesla, in comparison, had 182 disengagements but only 550 autonomous miles. Breaking this down a little more — a Waymo autonomous car is going for over 5,000 miles before it needs to disengage, and Tesla has to kick back to a driver every three miles.
In a post about the disengagement data, Dmitri Dolgov, head of self-driving tech at Waymo, wrote:
We’ve been able to make dramatic improvements to our technology because we use each of these disengages to teach and refine our car (that’s why we set our thresholds for disengages conservatively). For each event we can create hundreds— and sometimes thousandsof related scenarios in simulation, varying the parameters such as the position and speed of other road users in the area. This allows us to do a more thorough job identifying the root cause of any disengage and resolving any problems in a robust way. Last year alone, we drove over a billion miles in simulation, with a focus on tackling some of the toughest situations people could encounter on the road.
In its statement to the DMV, Tesla reiterated that it performs autonomous trials on test tracks and public roads around the world, meaning the report only shows the testing it did in California. However, BMW drove about the same number of test miles as Tesla in CA — 638 — but only had one disengagement.
The metric that seems to be missing here, which Musk would likely be quick to point out, is that the 550 autonomous miles Tesla recorded in California surely does not include the 220 million miles Tesla vehicles have driven with Autopilot engaged. Autopilot is not defined as autonomous under California law, so this driving data isn’t being included in the CA DMV’s reports. Musk is testing the self-driving capabilities of his cars via progressive updates, while other companies, like Waymo, are taking more of a behind the scenes approach.
Tesla is far from the only company with an advanced self-driving system, but its testing plan seems different. Still, Waymo’s impressive 635,868 miles driven means the company is catching up fast, and pursuing a different development plan than Tesla, which makes the two hard to compare. But if disengagments during test driving are the metric the government is collecting and seems to prioritize, Waymo is absolutely crushing the competition.
