Inside the Breakthrough VR Tool That Blends Art and Life

Goro Fujita is currently an art director at Oculus Story Studio in San Francisco, helping the Facebook-owned company develop a new kind of interactive art experience. With over a decade working in the animation industry as a visual development artist, Fujita is applying his experience in visualizing worlds, characters, and creatures into the uncharted new frontiers of virtual reality.

Fujita spoke to Inverse about developing immersive VR, as well as the many applications that are used to produce it. In particular, he discussed the design of Quill, the production app that allows artists to paint and have their artistry be rendered in three dimensions for viewers. That allows viewers to virtually “step inside” the painting, an experience used to great effect in the new short film Dear Angelica, which premiered at the Sundance Film Festival.

What were the technical and/or artistic challenges that led to the development of Quill?

Our Creative Director Saschka Unseld had the vision to create a narrative VR experience entirely through VR illustrations. The big question was if it was possible to immerse the audience into an artist’s creation. We did several different attempts where we mapped illustrations as textures onto 3D geometry or let particles form the lines of an illustration, but all of our tests looked more technical than artistic, meaning you could tell that it was engineered.

We had to find a way to let artists directly illustrate in VR. That’s when Inigo Quilez came up with a VR painting tool prototype that became “Quill.” It allowed the artist to paint directly in VR without any technical steps in-between, where the end result was 100 percent the artist’s creation. Saschka then hired the talented artist Wesley Allsbrook as the artist for Dear Angelica. Her art had the right energy and feel for the project.

What are the benefits for designing directly in VR as opposed to the traditional ways you’ve approached designing characters/worlds in 2D for animated films?

Ideally, if you design for VR, you want to design in VR. The reason is because your perception on a 2D screen vs. your perception in VR is vastly different. So, with the traditional approach, you lose a lot of time iterating, as you always have to wait until somebody ports what was designed in 2D into a 3D VR environment. Quill enables the artist to design freely in VR without any limitations. You get a sense for the scale and feel of a character, set, lighting etc., which is invaluable for VR productions, as early on you can make important design decisions without diving into a whole production process.

But it doesn’t end there. These assets painted in Quill can be easily ported into tools like Unreal as 3D geometry and used for pre-visualizations or even final assets, as well.

What are the differences between Quill and Medium? How have they influenced your artistic process? What excites you about these tools as an artist?

The main difference is that Quill is a painting tool and Medium a sculpting tool. An easy comparison would be Photoshop vs. ZBrush. In Quill, there is no lighting, and you have an infinite canvas. Although you are in 3D space, my work flow is very similar as when I illustrate in 2D painting applications. The infinite canvas enables you to create huge worlds or even worlds within worlds.

When I sculpt in Medium, my approach is very different, as I treat it like sculpting in clay. Lighting is given so I can focus on form and local colors. You also send your sculpts directly to a 3D printer. It’s extremely rewarding to hold a physical sculpt of your digital creation in your hands.

Both tools are incredibly intuitive and bypass all the technical barriers that you can run into in traditional approaches. The possibilities with these two tools are truly limitless, and I cannot be more excited about the future of art in VR.

You made an animated GIF of a series of sculptures of a rabbit walking, which you created in Medium. What do you think is the potential for storytelling (e.g. storyboarding, animation, stop motion) in this new VR art form?

It is inevitable that animation will be a huge part of tools like Quill and Medium. Quill already opened a door to a new art form with what it is currently, but with animation support it will be another game changer. It will influence the entertainment industry in a big way and change the way we experience art forever. At Story Studio we are already using Quill and Medium in our production pipeline, and I have no doubt that they will become key design tools for VR.

What do you think the future holds for artists working in VR? What impact do you see these tools having on the film, animation, and/or games industries?

I think the impact will be incredible! Art in VR is powerful in a way that you can reach people in ways it wasn’t possible before. I looked at art all my life, but I never felt truly connected to a piece. But I will never forget the first time I saw the New Sang Chong Market](https://www.facebook.com/oculusstorystudio/photos/a.1524694161129054.1073741828.1523983157866821/1776866949245106/?type=3&theater), a Quillustration (so we call it) by Carlos Felipe Leon, where I was hit by emotion and started to tear up without knowing why. I felt truly immersed in his imagination and did not know how to handle the emotions I was hit by. That was the moment when I knew that we found a new art form.