How the Lord of the Rings VFX team brought Tolkien's greatest monster to life

In a trilogy full of heart-in-your-mouth moments, one scene from Lord of the Rings stands out as particularly gut-wrenching.

Near the climax of the first film, The Fellowship of the Ring, we come face-to-face with a creature whom Gandalf describes as “a demon of the ancient world”: a fiery Balrog who threatens to prevent the fellowship from escaping the Mines of Moria. As the band of hobbits, humans, and elves flee — “Swords are no more use here,” Gandalf tells Aragorn — the wizard stands his ground against this ultimate foe, Durin's Bane, a colossal, minotaur-type beast with wings and weapons aflame.

With perhaps the most iconic line in the nine-hour saga — “YOU...SHALL NOT...PASS!” — Gandalf appears to vanquish the Balrog. He turns to join his comrades. But just as the audience is exhaling en masse, the creature's whip lashes out at Gandalf, pulling him into the chasm below. The seemingly immortal wizard is suddenly a great deal more mortal. For a moment we are all Frodo, screaming “No!” at the top of our lungs.

Weta is the New Zealand team behind this sequence, along with many others in recent years. Twenty years after the Fellowship was released, four members of the Oscar-winning special effects studio tell Inverse about the challenges and joys of breathing life into the Balrog.

A new challenge

Lord of the Rings wasn't Weta's first project, but its scope made it by far its biggest. The company was founded by Peter Jackson and colleagues in 1993, and the work on the Balrog was concentrated around the turn of the century, following films like Contact and Heavenly Creatures. Animator Andrew Calder was in his early 30s when he came on board, and Lord of the Rings was the first film he had ever worked on. He compares this challenge to never having climbed a mountain before and being asked to scale Everest.

“Peter was pretty clear, I think, on how to make it ominous and really scary,” says Sandip Kalsy, a technical director at Weta. By the time the team came to the Balrog sequence, the actors had all done their part, running around chasing imaginary bridges and ghouls. Kalsy says that for the actors, a man will often hold a stick with a green ball on it to represent the eyeline of the creature. “Then the man with the stick probably runs after them.”

Near the end of production on Fellowship, Stephen Unterfranz was working in a department known as “Creatures.” This team of about 10 people would work on the Balrog once the workshop team had made a physical model of it. For this to happen, the three-dimensional model needed to be taken into a dark room to be laser-scanned. In this case by a scanner that had originally been used to measure water loss in beef in walk-in freezers. That scan would then be turned into a digital model comprising mathematical representations of 3D geometry called NURBs (non-uniform rational B-splines). At this point, Creatures would get its hands on it, building the internal anatomy of the beast: the digital skeletons and muscle systems from which the animators could work.

One of the challenges, Unterfranz remembers, was the Balrog's size and the effect this had on how quickly it ought to move. “Any creature over a certain size is nothing we've ever experienced in real life,” he says, “and so when something that big falls down or takes a step, it looks ridiculously slow.” But moving slowly wasn't too big of a problem, says Calder, because the Balrog “has a sort of stateliness about it”. As a character, it needed to possess “majesty and heft.” The confrontation between the demon and Gandalf had to be framed as spiritual, not physical, he says, because Gandalf would not have survived a simple fight against the Balrog: “It just steps on him and he's gone.”

Playing with fire

In animating the Balrog, they needed to start as realistically as possible, Unterfranz says, so that they had a foundation to work from. Though the Balrog isn't a real creature (as far as we know), it needed to be grounded in the laws of physics and biology. This meant the team wasn't just “wandering in the wilderness” when making decisions about how the character would move.

Having created a virtual model, Creatures would send it to the animation department, where it arrived in a crude, greyscale form. (When he first saw it, Kalsy giggled at how unthreatening it looked.) Calder remembers working on the moments from when the Balrog pulls out its whip to when the bridge collapses under it. The whip, a piece of virtual geometry that Calder had to guide through the air with a mouse, took a long time to animate. He likens its motion to drawing with a sparkler, in that the team applying the fire to the whip knew it shouldn't ignite all at once but rather leave a trail of fire as it traveled.

“They kept going, ‘Faster! Faster!’”

Computational fluid dynamics — the technique that would be used to simulate fire today — was in its infancy, but Kalsy says the Balrog was simply too vast for this technique to work. Producing fire on that scale was a huge challenge, and the Balrog sequence alone took months.

Where there was fire on the Balrog, there also needed to be heat haze, and this was Kalsy's specialty. He needed to research how heat haze actually works. “I remember at one point my head was swimming with all the physics,” he says.

In order to simulate the haze, he would animate particles coming off the Balrog's body, allocating them red if the haze traveled right and green if it traveled left. These “cards” would subsequently be interpreted by a compositor. “The way that the red and green would wobble is what would actually influence distorting the image to make it look jittery and wavy,” says Kalsy.

Working as a team

Throughout, the working environment at Weta was one of a small team working long hours in close and friendly proximity. They were in a building in Miramar, Wellington, the suburb where Jackson grew up. “Back then, doors were open and you'd go crashing into people's offices,” says Kalsy. Calder says communication was always face-to-face. “If you wanted to talk to someone, you'd just stick your head out the door and yell, ‘Round the corner!’”

Because animation relies on a sophisticated understanding of how bodies and objects move, it is not uncommon for animators to try practicing the sequence's motions using their own bodies. “You often come ‘round a corner and find an animator down on all fours crawling around,” says Calder. Jackson and his “entourage of note-takers” would visit the team every few days, says Calder, and Jackson himself would also act out the Balrog scene in his bare feet. His reactions to their work were almost always jovial, according to Calder. “He's always been pretty good with the artists. He knows that it's a painstaking process.”

“Oh, we've got something special here.”

When it came to Gandalf plunging down the chasm with the Balrog — the beginning of The Two Towers — the effect looked “weird,” remembers Unterfranz. The various departments convened, and senior visual effects artist Joe Letteri asked them to imagine how the fall would look if it were happening realistically.

“If we said that the chasm is 100 meters wide and two kilometers deep, and they are falling at 9.8 meters per second squared and the camera's following them,” says Unterfranz, “what does it look like?”

They realized that if they portrayed it faithfully, the Balrog would fall too slowly, so they would need to “cheat” its speed a little. To convey the impression that the walls were rushing past the pair, says Calder, they ended up choosing a speed that would in fact have broken the sound barrier. “They kept going, ‘Faster! Faster!’”

Compositor Paul Conway, who traveled to New Zealand from the UK on a two-year contract and never left, didn't start working on the Balrog sequence until the spring of 2001. He says Jackson wanted the scene to remain realistic and not succumb to the risk that it might have a “heavy metal” feel. Conway's job was to take the various layers — actors, Balrog, etc. — and make it look as though they were “a complete, integrated environment.” To do this, he worked closely with Kalsy and people in the computer-generation department.

“I think film work in general, and visual effects in particular, is a collaborative process,” Conway says, “because one person will create one aspect of the shot, and they'll hand it to the next person. The whole company was dedicated to this one goal.”

Balrog unleashed

Calder remembers watching part of the scene, with all of its effects and music, after it had gone down well with an audience in Cannes. “I think even up to that moment we weren't really clear whether we were gonna pull it off,” he says. But this footage blew away any doubt: “Oh, we've got something special here,” they thought.

Conway recalls seeing it on the big screen in Wellington's Embassy Theater after the rest of the film had been finished as well. It looked to Conway like a new era for fantasy filmmaking. “To hear it with a full surround sound environment and the whole ground is shaking...it was pretty mind-blowing,” he says.

The Balrog sequence represents filmmaking in one of its most collaborative forms, and Weta was well on its way to becoming a well-oiled machine. In visual effects, it’s impossible to create such spectacular effects without many heads coming together to come up with solutions.

“You need to know that other people are there to help if you need it,” says Conway. “And vice versa — you offer your help to people as well. You're in it together; that's the cool thing about it.”

Note: In the Dream Teams spirit, the Weta Digital team wanted to call out the enormous contributions to the Balrog of Stephen Coren and Gray Horsfield, who were unable to be part of the interview.

DREAM TEAMS IS A SERIES FROM INVERSE that takes a look back at the greatest team efforts of the 21st century and what they mean for our ability to collaborate in the future.