What is LIDAR? Apple's new iPad Pro technology, explained

Apple launched a new iPad Pro Tuesday, and it features an all-new sensor. The company's latest consumer product is the first to feature a lidar sensor – although it's not the company's first project to use the sensor – and it's set to enable all-new augmented reality and 3D sensing features.

The new tablet is the most recent upgrade since Apple released the third-generation iPad Pro in October 2018. The new model features the same button-less front design with Face ID, with an 11-inch screen starting at $799 and a 12.9-inch screen starting at $999. It has an A12Z Bionic chip, a second rear camera for ultrawide photos, better microphones, and the previously-mentioned lidar sensor.

It also supports a Magic Keyboard accessory, which adds a trackpad that enables a cursor. The new keyboard has cantilevered hinges for viewing angles up to 130 degrees, and a backlit keyboard with scissor mechanism (goodbye, much-hated butterfly keys). The keyboard has one millimeter of key travel, an improvement from the 0.7 millimeters found on the controversial 2015 MacBook keyboards and one that should make typing feel more satisfying. It also has USB-C pass-through charging and a design that Apple claims will enable full "lapability" -- in that you can rest it on your lap. The features all set the stage for a showdown with the Microsoft Surface.

But it's perhaps the lidar sensor that's the fascinating part of the new tablet.

What is lidar?

Lidar stands for light detection and ranging. It dates back to the early 1960s when laser research was improving. It works by firing a light at an object, then measuring the time it takes to return. Because the system knows the speed of light, it can work out the distance by measuring the time. Velodyne explains that most lidar sensors these days use either 905-nanometer or 1550-nanometer wavelengths. Visible light, by contrast, measures between 400 and 700 nanometers.

Autonomous car developers are using lidar to measure object distance and help the car's cameras. Waymo, for example, uses four lidar sensors on the bumper and two sensors on top of the vehicle. Tesla is notable as it is not using lidar in its autonomous car development project.

Apple has been spotted trying out lidar with its "Project Titan" autonomous car project. It's also been rumored to be including a time-of-flight sensor in this year's high-end iPhone models, which would similarly bounce lasers off objects to measure distance.

How will lidar work on the new iPad Pro?

The new lidar scanner can measure an object up to five meters (16 feet) away. It's capable of running indoors or outdoors, and Apple claims it can operate "at the photon level at nano-second speeds." The new A12Z Bionic chip uses computer vision algorithms to make sense of all this nano-second data.

The new system leverages "ARKit," a toolkit first released in 2017 with iOS 11. The new kit enables developers to build augmented reality apps quickly. A new Scene Geometry application programming interface allows developers to take advantage of the lidar sensor specifically.

What does this mean for apps? Expect improvements to the Measure app used to calculate the size of objects using the camera. The app now also features a Ruler view for more fine-tuned measurements.

Third-party developers have already started trying out the sensor. The IKEA Place Studio Mode later this year will enable users to furnish their entire room with augmented reality. Updates to the Complete Anatomy app can help physical therapists track their patients with a new mobility assessment tool. Other uses include an update to Shapr 3D to create a 3D floor plan, and a new augmented reality mode for Hot Lava, where the floor really is lava.

What does it mean for iOS 14 and iPhone 12?

The new update gives a glimpse of Apple's plans. The company is expected to release four new iPhones in the second half of 2020, two of which will feature a new time-of-flight sensor that could work similarly to the new iPad. Apple is also expected to include new augmented reality features in iOS 14, including an app called "Gobi" that allows users to preview products in a store.

But in the far future, Apple is rumored to be developing a set of augmented reality glasses that work alongside the iPhone. The iPad Pro could be the predecessor to an all-new way of experiencing the world.