iOS's Animojis Could Soon Be More Interactive Than Ever, Patent Reveals

Apple’s Animoji and Memoji features leverage the iPhone’s FaceID sensors to transform your face into an animated cartoon lion, a poop, or robot. Building on the often viral new tech, users might soon be able to use speech or music to further interact with their animated doppelgängers.

The Cupertino tech giant seems to be taking a page out of Snapchat’s playbook and having Animoji and Memoji characters visually react to certain words, phrases, and even songs. A patent application titled “Voice effects based on facial expressions” filed on February 28 and published by the United States Patent and Trademark Office on November 22 reveals that users will be able to add effects after an Animoji message has been recorded.

It’ll work similarly to how certain Snapchat filters react to certain winks, nods, or waves. iPhones will track for both visual and audio cues instead of just mirroring the users’ face as closely as possible. This will then let people add in visual or auditory effects to their character based on their facial expression, what they said, or what song they were playing.

A diagram in the patent demonstrates a simple example of when a user chooses to record a message using the dog Animoji. Every time they say “Hello” the user can go back and change that word to “Woof.” This could work to add vivid graphical effects, like confetti every time you say “Congratulations” or a firecracker sound effect when you say “Boom.”

The flowchart suggests that each Animoji character will have an “avatar specific word,” meaning the unicorn would nay and the animated poop would theoretically fart (or something).

Apple has seemingly teased these editing capabilities in a previous commercial. The company created a short Animoji music video for the Migos track “Stir Fry” where the background constantly changes based on what the animated characters say.

All of these effects were likely added in post production, but they seem to give a pretty clear picture about where the efforts behind this patent suggests are headed, the gradual evolution from primitive emoji to more fully fleshed out digital avatars. For example, roughly ten seconds in the animated puppy says, “Money changin’ colors like tie-dye” which prompts currency to appear in the background.

Could personalized Animoji music videos be in Apple’s future? It’s not that farfetched, particularly as augmented reality software gets better at responding to aural and visual cues.