Tech
There's an iOS app that inserts your body into VR games
The Mixed Reality app will place your body into the action of games played through Oculus Quest or Quest 2.
A yet-to-be-released app called Mixed Reality takes advantage of the ARKit augmented reality framework in iOS to insert VR gamers directly into their gameplay videos. Accomplishing this otherwise would require recording oneself with a camera and using a green screen to splice themselves into gameplay footage afterwards using arduous video editing software. Most gamers who stream online use picture-in-picture to show the gameplay alongside a shot of them using their VR gear.
A good use of AR — When it goes live (should Apple approve the use), Mixed Reality will work with the Oculus Quest or Quest 2, all-in-one VR headsets that can connect to an iOS device via Bluetooth. Also required is an iPhone or iPad that features an A12 Bionic chip or newer. It looks like gamers will point their iOS device's rear camera at themselves as they play and the app will do all the work of putting them in the action.
Apple has spent years developing its AR framework to recognize the contours of objects in an environment so they can be captured as 3D objects and manipulated digitally. LIDAR in the new iPad Air and iPhone 12 Pro improves on Apple's software work by adding infrared lasers to more accurately recognize the depth of objects in an environment. The developer behind Mixed Reality says it uses the "person segmentation" feature in ARKit that has been specifically trained to recognize people so they can be manipulated in different ways.
Virtual reality goggles like the Oculus Quest are the opposite of AR in that they place gamers into a captive digital environment. But they also take advantage of AR by using cameras to map the shape of a room so players are alerted when they're about to run into an object, and to track the position of the controllers for gameplay.
Apple's AR interests — There have been rumors for years that Apple is planning on developing AR glasses, and its investments in the sensors and technology suggest it's possible if the company can get the hardware into a slim enough package.
Facebook is developing AR glasses and says it's been able to squeeze all the gadgetry into a pair of Ray-Ban glasses.
Apple CEO Tim Cook has said he thinks that AR could be the next major computing platform after the smartphone as it could enhance communication. Not many applications have yet been developed for AR besides fun face filters, however — and Apple's cute Memoji avatars. But if it really is the next big leap in computing it's no wonder the tech giants are racing to be first.
Mixed Reality is probably one of the coolest uses of AR thus far in iOS, and could make it really fun to record gameplay to upload to YouTube or stream to Twitch without needing to do much heavy lifting. If you're interested in signing up for the beta, there's a Google form here.
