It’s fitting that in a year when the Visual Effects Society presents Martin Scorsese with its Lifetime Achievement Award, the director’s latest release, The Irishman (Netflix), is breaking new ground, arguably most notably in the effects arena. The Irishman is nominated for the VES Award for Outstanding Supporting Visual Effects in a Photoreal Feature.

The nomination marks the seventh thus far in the career of VFX supervisor Pablo Helman of Industrial Light & Magic. Helman won in 2006 for Best Single Visual Effect of the Year for War of the Worlds. The Irishman is the second Scorsese film for which Helman earned VES nominee status--the first coming in 2017 for Silence.

It was back when working on Silence in Taiwan that Scorsese first approached Helman about how to best go about attaining a certain holy grail of effects--the de-aging of actors so that they appear decades younger.  Over Thanksgiving dinner during a break in the shooting, Helman recalled, “We started talking about how to make an actor younger. That’s when the director told Helman about The Irishman, getting him a script that very same evening. “In the morning, I said, ‘Yes, I want to do it,’” said Helman.

Such a technical/creative fountain-of-youth discovery would be integral to realizing The Irishman--specifically turning back the clock for actors Robert De Niro, Joe Pesci and Al Pacino.

One one hand The Irishman is an epic saga about organized crime in America, taking us from post World War II through the Kennedy administration and beyond. But ultimately the film is a character study--with dramatic and wryly humorous elements. 

Through its protagonists, The Irishman shows us not just a life lived in and around the mafia but the toll that life takes on a lone person now that he has the time to reflect on it in his old age--and in particular the melancholy he feels over the lack of a relationship with his daughter. 

It’s a tour de force performance by De Niro as Frank Sheeran, the Teamster and mafia figure who claimed before his death that he had killed Jimmy Hoffa (portrayed by Al Pacino). Also at the core of the film playing a masterful role in Sheeran’s life is crime boss Russell Bufalino (played by Joe Pesci), whose persona alternates between sinister and sympathetic.

Scorsese has called de-aging that doesn’t compromise the actors the next evolution of makeup. Helman noted that it’s been something that visual effects artisans have been looking to do for years, a brand of effects that supports the content of a film in a very important way. 

In the past, de-aging involved facial markers, helmet rigs, garb with sensors and other encumbrances that got in the way of the acting. Scorsese wanted youthful versions of his actors sans any physical impediment to their live performances. This would free the auteur filmmaker to fully tell the story of The Irishman.

The collaborative process between Scorsese and Helman started four years ago, with two years focused on inventing a new piece of software that would help to capture performances of actors sans any physical encumbrances. Complementing that software is a three infrared camera rig that allows triangulation, creating a kind of 3D geometry of and around the actors; the center camera being the “director’s camera” flanked by two other “witness” cameras. This enabled Helman to eliminate shadows created by on-set lighting, shadows that could potentially interfere with the geometric facial shapes constructed by de-aging software.

Helman and his team then spent a couple of years sifting through old movies of De Niro, Pacino and Pesci, cataloging the targeted ages for each of the respective actors in The Irishman. They created a program where one actor’s face is swapped for another’s to help check that their work on the movie was on the right course, with the system generating hundreds of images for cross-referencing.

Actors could thus act younger on camera and then the technology would later make them look younger. 

Helman noted that the burden thus is taken away from the actors--and in some respects transferred to the production designer. For example, doors on set would have to be 32 inches wide so that the camera rig could pass through them. Also in The Irishman there was a problem presented in the use of vintage 1950s and ‘60’s automobiles which had lead in their windshields. Infrared can’t pass through the lead. So the windshields had to be removed from the car and put back via CG.

But all the logistical issues are worth the end result, assessed Helman. “The performances are so incredible. The actors were allowed to as truthful to the script, dialogue and scenes as possible without having to worry about what we were doing.”

Helman has come to deeply value his working relationship with Scorsese. The two first met on a location scout in Taiwan in 2014 for Silence. “We hit it off,” said Helman. “He is an incredible filmmaker with an incredible eye for framing, timing, sequencing, rhythm. He’s very intuitive yet very knowledgeable about film. It’s also been an incredible opportunity to learn from him. He’s very collaborative, always ready to listen and very funny.”

Helman has learned from Scorsese and from his experience of working four years on The Irishman. Helman acknowledged that The Irishman was “an incredibly risky project from the beginning but we never lost hope. I never felt we weren’t going to be able to do it.” His confidence was rooted in the team at ILM, one he grew up together with over the past 24 years.

While he’s gratified over the finished film, Helman started to feel that sense of satisfaction earlier on, “working three feet away from De Niro and Pesci. You see their eyes are connecting,” related Helman. Seeing that interplay between actors, the freedom to innovate and ad lib, meant the world to Helman who said that The Irishman gave him more insights into human behavior, performance and art.

Helman’s filmography also includes the documentary Rolling Thunder Revue: A Bob Dylan Story by Martin Scorsese. Beyond the VES Award recognition he’s received over the years, Helman is a three-time Best Achievement in Visual Effects Oscar nominee--for Star War: Episode II--Attack of the Clones in 2003, War of the Worlds in 2006, and now for The Irishman. 

Robert Legato
Another effects artisan--this one the recipient of an HPA Lifetime Achievement Award in November--also has a history with Scorsese. Visual effects supervisor Robert Legato earned one of his three VFX Oscars for Hugo, a collaboration with Scorsese. Five of Legato’s 10 career VES nods are for Scorsese films: two apiece for The Aviator and Hugo, and one for The Wolf of Wall Street.

This year, though, another prime collaborator, director Jon Favreau, figures in Legato’s VES showing--yielding a pair of nods for Outstanding Visual Effects in a Photoreal Feature, and Outstanding Virtual Cinematography in a CG project on the strength of The Lion King (Disney). Both The Lion King and Alita: Battle Angel lead this year’s field of VES nominees with five nominations each.

Back in 2017, Legato won a pair of VES Awards for another Favreau-directed feature, The Jungle Book, for which Legato also earned a VFX Oscar. And now The Lion King has yielded Legato’s fifth career Academy Award nomination.

Like Scorsese, Favreau has earned a VES Lifetime Achivement Award, his coming back in 2018. And like The Irishman, the effects work on The Lion King was groundbreaking. The ensemble of talent on the film combined meticulously researched character and set design to render each scene within a VR environment that spanned hundreds of (virtual) miles using the Unity video game engine. Every piece of this rich world was crafted by the VFX team. The characters themselves took about six months to create--hair, muscles, facial expressions, all of it carefully crafted to both ground this story in photorealism while maintaining emotional resonance. Meanwhile, the team also had to translate James Chinlund’s production design concepts into credible African landscapes, dotting them with authentic vegetation and boulders and termite mounds, crafting environments that are at once visually distinct and cohesive as a whole, evocative of the classic designs but something all their own.

Putting everything in place in that way gave Oscar-nominated cinematographer Caleb Deschanel and the rest of the filmmakers the opportunity to use traditional filmmaking skills--setting up shots, choreographing movements, adjusting lighting--to tweak the scene in perfectly real time, all with the use of a VR headset on a digitally rendered African savanna. 

Favreau’s layered approach to making the film included a blend of traditional live-action filmmaking techniques, state-of-the-art virtual-reality tools and the highest-level CG animation. The end result is a wholly believable, photoreal look that transports moviegoers to the Pride Lands.

MPC Film was an integral part of the process from the beginning, spearheading the visual effects. MPC’s VFX supervisors Adam Valdez (part of the Oscar-winning visual effects team for The Jungle Book) and Elliot Newman helped plan approaches on how the movie could be made. Their virtual production team worked with the filmmakers to develop The Lion King’s virtual production technology.

Everything ultimately seen on screen was created in the computer, but it is anything but traditional animation. Favreau explained, “Where we departed from animation--beyond the photoreal look--was, at the point when you would normally operate the cameras in layout on a computer, we stopped the process and brought the entire film into VR and let our live-action crew actually set up real camera equipment.”

Legato said of the unique approach, “People are studying animal reference and the animators breathing their life into these digital rigs. So, we’re taking an antiseptic digital medium and telling one of the most emotional stories that we have in our tradition using these tools. That dichotomy and underlying tension creates a lot of creative opportunities. This is as close to practical filmmaking as you get with an animated film.”

Filmmakers kicked off production with a pre-visualization (pre-viz) phase commonly used in animated filmmaking. Animation supervisor Andrew Jones and the team of artists created simplified animated sequences so that it could run in real time in VR. These early versions of environments and characters became part of the Unity gaming system. Favreau said, “Instead of watching it play on the computer screen, we could go into the environment and stand next to an animated lion.”

The virtual production employed in The Lion King is an extension of what was done on The Jungle Book. Favreau and his team were able to don VR headsets and walk around within the virtual set, setting up shots, choreographing movements, and adjusting lighting, characters and set pieces in real time before sending the version of each scene to editorial.

The idea behind incorporating live-action language into the film was to convince audiences that what they’re seeing is authentic. The data obtained during the virtual production was utilized by the animation team. Scenes and recordings were exported to editorial as video files, and to visual effects as data files that gave clear direction to the visual effects crews around the world who crafted the film’s photoreal aesthetic. Preserving the invisible hand of the filmmakers throughout maintained the film’s live-action style.

Once character designs were approved, artists from MPC built each character within the computer, paying painstaking attention to anatomy, proper proportions, fur or feathers--applying textures and color, shading eyes and ensuring their movement was authentic to their real-life counterparts. New software tools were developed by MPC R&D’s team of more than 200 software engineers to better simulate muscles, skin and fur.

In all, the London-based MPC Film’s VFX artists brought 86 different species to life for The Lion King--from the film’s iconic characters like Simba, Nala, Rafiki, Mufasa, Pumbaa and Timon, Scar and the hyenas--to the smallest creatures on the savanna.

VES Award nominees in 25 categories were selected by VES members via events hosted by 11 of the organization’s Sections, including Australia, the Bay Area, Germany, London, Los Angeles, Montreal, New York, New Zealand, Toronto, Vancouver and Washington. The VES Awards will be held on January 29 at the Beverly Hilton Hotel.  While Scorsese will receive the VES Lifetime Achievement Award, the VES Visionary Award will be presented to director-producer-screenwriter Roland Emmerich. And the VES Award for Creative excellence will be presented to acclaimed visual effects supervisor Sheena Duggal. Full coverage of VES Award winners will appear on SHOOTonline and in the January 31st SHOOT>e.dition.