'Harold Halibut' art director reveals the secrets to making games with clay

3D models are the standard for video games.

But what happens if you want to make a game but lack the industry-standard digital crafting abilities? Well, you could spend years learning how to manipulate 3D models or you could try something else. That’s how the German-based game studio Slow Bros. spawned their first major game, Harold Halibut.

The story follows a human colony in the far future that has been submerged in an alien ocean on a city-sized spaceship for the last 250 years. The game follows Harold, an assistant to the ship’s lead scientist. Harold, his boss, and every other character in the game was made completely by hand. Every character and environment was made out of clay, giving their game a uniquely real feeling.

While clay models have been standardized in films for decades with modern darling stop motion-centric studios like Laika and Aardman, gaming is an unrefined space. Games have yet to normalize making physical models. Harold Halibut’s team had to create many of the standards for making a clay-based video game.

Inverse spoke to Harold Halibut Art Director Ole Tillmann at Tribeca Film Festival about what it took to craft scenes for the game, how their lack of skills forced them to innovate, and which stop-motion films inspired the game.

Can you take me through the process of making a single scene?

We’d start by fantasizing and discussing a place then write an outline. I’d sketch the scene. After double-checking a concept with the team, I’d meet with our set architect, Fabian Preuschoff. He’d build the set at a 1:10 scale from materials ranging from children's clay to welded metal to aged wood. Then it would be painted, “aged“ via chemicals that create rust and other patinas, dirt/dust smeared on the wall, etc.

The final product would be 3D-scanned via a process called photogrammetry. It's where a bunch of pictures are taken of the object so they can be assembled into a 3D model. Light, atmosphere, and VFX are set up inside this 3D scene in the game engine Unity and according to the physics at a miniature scale. We’d then place the characters and objects they’d interact with into the room. Characters are animated via motion capture and objects are either interactive or are pre-animated by hand in 3D. Our last step saw us adding voice acting, dynamic sound design, and music.

You made physical models for everything. Why choose to do this rather than copy a handmade aesthetic as games Little Big Planet and It Takes Two do?

This started out with a lack of technical knowledge about the alternatives like 2D animation/drawing or 3D animation, etc. A handmade game was the only linear storytelling method we had available. The further we developed this plan of “hand building“ a video game, though, the more we realized the potential, even beyond the look.

For example, the method of working itself … we were able to chemically age metal, let dust collect on the surfaces, wet stuff, physically break things, etc. Also, there’s a different sort of commitment without an undo option. There’s this immense richness of detail in our photographic textures of these natural surfaces in general. There’s an indescribable quality where people can recognize that other people have touched all these things, both characters as well as all the objects. Like sometimes it may be a literal fingerprint left on a wall.

What difficulties did you run into by choosing to make physical models?

Oh, plenty! Each new character-puppet we finished would present a new set of difficulties. Just putting a long coat on one of our characters forced us to figure out how to animate it or apply some sort of physics rule set to it.

We also ran into certain characters wearing items that were impossible to scan like multiple pearl necklaces. When that happened, our character artist would need a few days to re-model it in 3D using the physical necklace just lying on her desk.

Generally, we’d drop things and they’d break! We could’ve probably employed a miniature-finger-back-in-place-gluer. It was also tough to not have precursors for the process. Being first, also means you have to figure out a lot.

Despite our troubles, It was well worth the effort.

Stop-Motion seems like a big influence on the game, so which one inspired Harold Halibut?

First, the stuff we watched as children, which includes the Sandmännchen (Sandman), and a few other children-specific programs made in eastern Europe as well as Polish, Czech, and Hungarian film making (a lot of which we re-watched and looked more into in film/art school). There was a show called Pingu which I don’t know where this aired but it’s a cute claymation penguin that has since become a meme.

I remember seeing a lot of movie special effects made via stop motion, both Dinosaurs, King Kong, all the ILM Star Wars stuff. Wasn’t some of The Terminator even still stop motion-based? Eventually, there was Celebrity Deathmatch on MTV, which I remember vividly. Aardman (Wallace & Gromit) Robot Chicken, Wes Anderson and Laika films. If you haven’t seen it, look into Niki Lindroth von Bahr, a Swedish filmmaker … so good!

Do you have a plan to preserve any of the models from after Harold Halibut is completed?

It seemed ridiculous to toss them, so we’ve been preserving them sort of painstakingly, especially considering that we worked out of our living rooms for a good while, and then even our office could never hold the mountains of physical stuff we have accumulated over time. But we do fantasize about having an exhibition of some sort.

Harold Halibut is the first Slow Bros. game, correct? Will your next game use handmade assets as well?

Not necessarily. We’re no longer dependent on it. But we are also very much in the groove now and feel like we’re able to fully wield the power of this style, so it would be foolish not to. We’re also looking into using the technology for film-making as well.

Harold Halibut is currently in development.