Tesla Dojo: Why Elon Musk says full self-driving is set for 'quantum leap'

Tesla is training its car computers in a simulation, and Elon Musk claims it will lead to a "quantum leap" in autonomous driving.

Last week, the Tesla CEO declared via Twitter that "Dojo" — its neural net training computer — is "a beast." The company has been developing an improved version of its full self-driving system, with recent comments focusing on how the company has managed to feed data into its training system faster.

"Tesla is developing a NN training computer called Dojo to process truly vast amounts of video data. It’s a beast! Please consider joining our AI or computer/chip teams if this sounds interesting."

Musk also claims he can drive his car between his home and work "almost at zero interventions."

"The FSD improvement will come as a quantum leap, because it’s a fundamental architectural rewrite, not an incremental tweak. I drive the bleeding edge alpha build in my car personally. Almost at zero interventions between home & work. Limited public release in 6 to 10 weeks."

It's possible this could be a big step forward for the company's ongoing full self-driving project. In October 2016, Musk announced that all vehicles produced after that date would come with the necessary cameras and sensors to offer point-to-point autonomous driving. While the CEO claimed the company would demonstrate a coast-to-coast autonomous drive by the end of 2017, Musk later claimed that the results form such a system would have been "brittle." In October 2018, Musk wrote that it's “extremely difficult to achieve a general solution for self-driving that works well everywhere.”

If Musk's latest comments are anything to go by, "Dojo" could unlock the way forward.

What is Tesla Dojo?

Based on Musk's previous comments, "Dojo" appears to be a computer that Tesla is using to develop its autonomous driving system.

People expect autonomous cars to respond to a wide variety of unpredictable situations in sensible ways. Autonomous car projects can use neural nets to develop a system that can respond to new situations based on previous inputs.

Neural nets have been around for over 70 years; they essentially consist of feeding examples into a system to help train it up. An object recognition system, for example, might have been given hundreds of labeled pictures of cats and other animals. The system builds up an idea over time so that when a new image comes in, it can determine how close it is to previous images of cats.

Developers can also "train" an autonomous car system. This has led to something of a race between competing firms, keen to demonstrate their wealth of data.

Waymo, which started life as Google's autonomous car project, announced in March 2020 that it had completed 20 million miles on public roads and a further 10 billion miles in simulation. Tesla announced the following month that its cars had globally completed three billion miles in the semi-autonomous autopilot mode.

In April 2019, during Tesla's autonomy day, Musk mentioned the company has a training computer working in the background.

"We do have a major program at Tesla which we don't have enough time to talk about today called Dojo that's a super powerful training computer. The goal of Dojo will be to be able to take in vast amounts of data and train...at a video level...and do unsupervised massive training of vast amounts of video with the Dojo computer. But that's for another day."

Comments suggest that, alongside "Dojo," Tesla has managed to further automate this tagging process. During the same event, Tesla's head of artificial intelligence Andrej Kaparthy explained that the company has been using information from its fleet of vehicles on the road to automatically generate tagged data for its training systems.

Musk again mentioned the project in August 2019, when he, somewhat cryptically, wrote on Twitter "will Dojo to be the difference."

In February 2020, he expanded further by explaining that Dojo "will be able to process vast amounts of video training data & efficiently run hypersparce arrays with a vast number of parameters, plenty of memory & ultra-high bandwidth between cores."

What does Dojo mean for autonomous cars? – During the company's second-quarter 2020 earnings call, Musk explained that the biggest change that "really matters" for full self-driving is a move from analyzing isolated photos of a scene to one that analyzes video scenes in a smarter way. Musk likened it to a shift from "two-and-a-half D" to "4D."

Last week, Musk pointed to a Tesmanian article that explains the "4D" comment and describes a patent for improving the training data set. The process of manually labeling training data, needed to train up a neural net, is slow and boring, the article suggests. The company's patent explains how the car would capture information like speed, change in direction, and change in elevation. All of this information is used to inform the data sets that are built up, meaning "Dojo" can train with automatically-obtained information.

Right now, the company's "full self-driving" package that it sells with current Tesla vehicles promises limited autonomy in situations. One popular example is Navigate on Autopilot, which drives along the highway autonomously and turns off at the correct exit based on inputted destination.

During the earnings call, Musk claimed the new version would be able to do "traffic lights, stops, turns... everything pretty much." From there, the company would be working on a "march of nines" – getting the system to work in 99.9 percent of cases, with an increasing number of nines on the end.

When will Tesla offer fully autonomous driving? – That is the big question. Musk claims improvements will start making their way out into the public in the coming weeks. At last month's World AI Conference in Shanghai that he also said he is "confident" the team will have the "basic functionality" for fully autonomous driving this year.

Regulatory approval, which depends on politicians giving that system the legal thumbs-up to operate with no human oversight, could be further away.