Advanced Math Will Kill the Big-Nose Selfie
The problem with all those thirsty pics is single-point perspective. It can be solved.
Whether or not you want to admit it, selfies offer portraits of modern society. Sometimes those portraits are unflattering; sometimes they’re manipulated. A number of photo editor apps — notably Kim Kardashian’s beloved Perfect365 — allow Instagrammers to put their best, doctored foot forward. And no commercially available app currently allows anyone to wipe away the distortion that comes from taking an arms length look at oneself. Selfie sticks help (depending on how you look at it), but perspective is hard to come by when one is shooting up close. That’s where a plucky band of Princeton graduate students have found an opportunity. Using some advanced math, they’ve created a portrait photo editor that takes a less distorted selfie.
Ohad Fried, Eli Shechtman, Dan B. Goldman, and Adam Finkelstein are the four Princeton students who partnered with Adobe to develop software that manipulates photos based on varying camera perspectives. “The point of the project is that the distance between the camera and the subject really matters,” Fried explained to Inverse, referring to a previously published paper that came to this conclusion by surveying people about their associations with portraits taken from varying distances. The software allows users to adjust this crucial factor without actually altering their facial features.
With the new software, you can adjust the distance of the camera, yaw (side to side), and pitch (up and down)
According to the group’s research, portraits taken from closer distances can result in big noses, weak chins, and sloping foreheads. These traits are associated with terms like “peaceful” and “approachable.” Portraits taken from further away tend to more accurately display the proportions of a person’s face and make them appear more “attractive” and “strong.” By using the new software to adjust the camera’s perspective, users can make more precise efforts to make their selfies convey their desired qualities.
The differences between portraits taken from short distances and long distances are pretty major
Let’s distill the advanced mathematics that went into developing this program. First, the software detects landmark features, such as eyes, nose, mouth, and jawline, in a 2D photo so it can create a 3D mesh with those same annotated points. The point here isn’t to capture the person’s face, but to understand the picture. “It’s not just you, it’s you with half a smile, and it’s not just you with half a smile, it’s you with half a smile seen by a camera which is three meters away and not just a camera, an iPhone camera,” Fried says.
Once the 3D model is laid over the 2D photo, the software adjusts the location of the virtual camera to obtain before and after points, which are applied to the 2D photo to create the new version.
It’s only a two-step process. The steps are just very, very complicated.
Identifying the landmark locations on the 2D photo
Although this project has more to do with computer science than the selfie phenomenon, the results have inevitably pulled the team into the broader selfie conversation. Developing new portrait manipulation programs could potentially be viewed as perpetuating unrealistic beauty standards, but the team has isolated an important distinction between their software and other apps.
“Using Photoshop or whatever you can adjust pretty much everything on your face, and that includes taking your photo and turning you into someone else,” Fried said. “All this program is doing is showing you how you would look if you took the time to move the camera.”
Photos aren’t real, but Fried’s work is intended to create the potential to display something closer to what humans might see if they stood next to each other or made eye contact across a crowded party. These selfies put the self first.
The software can be used to make a selfie appear less distorted or to make a distant photo look as though it were taken from nearby
The perspective-aware portrait manipulation program currently exists online as a demo, although the online version is not as accurate as the real method the team developed. As a grad student working on his Ph.D. in computer graphics and computer vision, Fried is more interested in the research angle to the project than the entrepreneurial side of marketing this product, which would require a lot of additional engineering.
The future of the software is up in the air, but it looks like it may be divergent. Photo manipulation will help liars lie. Math will help everyone else tell the truth — or something a few inches from it.
