The Rush for Augmented Reality Patents Has Gotten Silly

There are a few frontrunners in the augmented and virtual reality race: Microsoft with its Hololens, Oculus with its Rift, the HTC Vive, and even PlayStation and Magic Leap with their unreleased headsets. As these companies proceed with their attempts to dominate the next generation of technological obsession, they’re scooping up intellectual property left and right. Each company is trying to patent anything that might be patentable.

This rush to cover all potentialities is smart. But it’s also amusing. The people behind these patents must be a particular brand of eccentric. If it were the 1940s, these brains would be spitting out pulp science fiction. Instead, they’re imagining the most futuristic possible applications of AR and VR technology.

Magic Leap is arguably the most exciting of these companies. It cherishes allure, and so we know next to nothing about the product. What little information we can find is buried in these patent novels. We’ve covered some of the more frightening possibilities in the past. But there’s plenty more we can choose from, and within these sprawling, tiresome documents are some true gems of the genre. Here’s a selection.

Unmoving Eyes Go Blind

User interfaces are what you use to interact with a technology. Magic Leap sees several potential virtual UIs. One could be a social media tree that a user plants, with each branch providing a particular media or action. Another is a spiderweb, employing much the same idea. The patent also outlines designs for pull strings, which the user yanks to trigger some action, a paint bucket, which the user splashes on a surface to provide options, and a finger brush, with which the user draws out a menu.

Exceptionally Long, Indecipherable Sentences Are Just Fine, Really

In another aspect, an augmented reality display system comprises an image capturing device to capture one or more images of a field of view of a user, wherein at least one image captures a manipulation of a predefined physical object, a display device to display one or more virtual objects to the user, a database comprising a passable world model, the passable world model comprising a set of map points associated with physical objects of the world, and a processor communicatively coupled to the image capturing device to recognize, based on the one or more images, a command to create a virtual user interface, to determine, based at least in part on the passable world model, map points corresponding to the predefined physical object, and to control the display in a manner such that the virtual user interface is generated at the determined map points corresponding to the predefined physical object, such that the virtual user interface appears to be stationary at the position of the predefined physical object.

Netflix’s Black Mirror Nailed It

In one or more embodiments, the virtual content is selected from the group consisting of information about the physical activity, the first user’s performance statistics, a virtual experience corresponding to the physical activity, and a virtual avatar.

While there’s no AR headset in this particular episode, the gamified exercise remains:

On a more frightening note, Black Mirror also predicted where the following idea would wind up:

If a registered user does not wish to be captured by other AR systems of other users, the system may, on recognizing the person, stop capturing images of that particular user, or alternatively, blur out visual images associated with the person.

On a more unsettling note: those who don’t wish to participate in the whole AR shebang may find themselves compelled to participate. For Magic Leap, that’s a good thing.

On the other hand, a person who has not registered with the AR system automatically has less control over privacy than one who has. Thus, there may be a higher incentive to register with the AR system (or associated application).

In VR, You Can Be Whatever You Want

Even if it’s distracting as all hell.

Based on the user’s preference, the avatar may be any image chosen by the user. For example, the user may render himself/herself as a bird that flies around the space of the conference room.

Leap Motion’s CTO David Holz Wasn’t Lying

Leap Motion is the foremost hand-tracking company for VR and AR headsets. Co-founder and CTO David Holz is pretty thrilled about the robotics-VR crossover, and it sounds like Magic Leap is, too.

In some implementations, the AR system may allow an avatar to operate a virtual machine, for example a virtual robot, to operate in an environment. For example, the AR system may render an avatar to appear to “jump” into a robot, to allow the avatar to physically change an environment, and then allow the avatar to jump back out of the robot.

Never Leave Your Home

Even today, agoraphobes have it pretty easy. Groceries, meals, laundry, supplies, products — almost nothing is out of reach for people who’d rather not get out into the world. But with an AR headset, agoraphobia will reach new heights. Users will be able to preview purchases — be it attire, decorations, or furniture — from the comfort of their own homes. Exciting concerts will no longer require people to exit their den: Magic Leap wants to stream concerts in 3D AR to whatever venue users prefer. So too for sports: people with AR headsets will be able to project virtual screens with alternate camera angles, statistics, and Twitter feeds onto the walls of their apartments. Or, better yet:

For example, the user may track an alternative route for a wide receiver. The AR system may make no changes to the actions of the players, except the selected wide receiver, the quarterback, and any defensive players who would cover the wide receiver. An entire virtual fantasy play may be played out, which may even produce a different outcome than the actual play. This may occur, for example, during an advertising break or time out during the game.

One Patent Author Was Pretty Tired While He Wrote

Because who doesn’t equate coffee with the Madagascan jungle:

When the user receives the cup of coffee that he has ordered, or upon detection by the system of some other pertinent parameter, the system may display one or more time-based augmented or virtual reality images, video, and/or sound in the local environment with the display device, such as a Madagascar jungle scene from the walls and ceilings, with or without jungle sounds and other effects, either static or dynamic.

This Sounds Downright Creepy

In yet another example, the system may include various forms of “super vision,” like telescope vision, see-through vision, infrared vision, God’s vision, etc.

There’s No Limit to Possible User Interfaces

User interfaces are what you use to interact with a technology. Magic Leap sees several potential virtual UIs. One could be a social media tree that a user plants, with each branch providing a particular media or action. Another is a spiderweb, employing much the same idea. The patent also outlines designs for pull strings, which the user yanks to trigger some action, a paint bucket, which the user splashes on a surface to provide options, and a finger brush, with which the user draws out a menu.

Flicka Da Wrist

Your hands will be the key to immersive and intuitive VR and AR experiences. But coming up with the right gestures is no small task, and patenting them is something else entirely. There are paragraphs upon paragraphs in some of these patents that detail just how a finger would touch another finger so as to yield a particular virtual interface. A few months ago, Leap Motion’s Holz shared some such concepts. There’s no limit: from the simplest gesture to the most complex, hands can control anything in the virtual space. Locking down those potentialities takes thousands upon thousands of words. These companies are racing to do so.

All Work and No Play Makes Jack a Dull Boy

VR and AR companies are pretty thrilled about what their headsets will do for workplaces. Mundane factory jobs could be gamified, as could, say, warehouse training courses. Employers could virtually “motivate” unproductive workers, using the headsets themselves to track productivity.

From the patent: “Because the AR system is constantly capturing images and videos, the AR system may detect unproductive behavior (e.g., unrelated internet browsing, low productivity, etc.), and may appropriately render virtual content to help motivate the user.)”

And these companies have imagined and patented many other augmented work experiences. Architects and industrial designers could concept in AR. Surgeons could control robotic tools or preview upcoming procedures. Et cetera.

Grocery Shopping Will Never Be the Same

Just two examples, though these patents cover far more hypothetical ground:

One or more virtual effects may cause a bottle of wine to recommend a cheese that goes well with the wine.

Number two:

The effects may include an image of a skinny version of the user, which is rendered in response to the user looking at a high calorie product.

This Could Go One of Two Ways

In another embodiment, the AR system may automatically display virtual content that has a calming effect on the patient. Or, in another embodiment, the AR system may be linked to a drug delivery system that may immediately administer prescribed medication whenever the patient displays a certain kind of behavior.

There’s no doubt that some if not many of these wacky ideas will make their way out of hypothetical land and into reality. Many of them — like gamifying employee training and exercise (which latter concept Pokémon Go proved) — are sensible. But it’s also likely that, like, for instance, iPhones, many truly novel AR advances will only happen once the headsets become commonplace. Until that day, the rush to scoop up intellectual property is sure to continue.