How Blade Runner 2049 created the most realistic hologram (and wildest threesome) ever

Contributed by
Oct 24, 2017

Blade Runner 2049 makes a number of grim predictions for our future. And while much of the technology developed in the film's dystopian world is used for evil — many replicants are slaves, after all — there are a few bright spots. Chief among them is the digital girlfriend, which allows lonely people to feel an interpersonal connection, even if it is simulated.

K, the detective character played by Ryan Gosling, is one of the many people who makes use of this new service, co-habitating with a hologram named Joi (Ana de Armas). And while it's clear from the beginning that Joi is synthetic, director Denis Villeneuve wanted to create a sense that she was more than just a souped up Alexa. One of the most important signifiers to audiences that Joi was "real" was, ironically, a property that actual humans most certainly don’t have: physical transparency.

VFX supervisor John Nelson and his team placed a batch of small cameras, including C3s and GoPros, to record actress Ana de Armas as she acted through each of her scenes with Gosling, so that they could capture her from as many angles as possible. All the different vantage points were necessary to create a volumetric version of Joi, complete with what Nelson calls a back shell, or "her back being seen through her semi-transparent front side."

"If you're looking at a glass and you put water into it, you see the front side of the glass but you also see the back side," Nelson told SYFY WIRE. "We mapped out what that back shell would look like, as if you had a bottle that had a label on both sides. It’s as if you look through the front of the bottle and see the label on the back of the bottle, but backwards."

The data from all the angles allowed them to position Joi exactly where Armas had stood during the scene, while creating a back shell that not only made physical adjustments, but could also act as a real-time, semi-transparent window to points beyond. Technically, all of that was possible to do without capturing any images at all — anything can be done in CGI these days — but the idea was to create a character that someone in the room could fall for, someone miles from the uncanny valley.

"She looked so real," Nelson said, "because she was made from a photograph."

 

Unlike the CG re-animation of Sean Young, another one of the film’s VFX feats, the goal wasn’t to create an entirely lifelike character. Joi is a hologram, after all, a piece of technology meant to suggest humanity but not take on its physical burden. Villeneuve used several different ways to ensure that people remembered that, and thus understood the conventionality of K’s relationship.

Right away, Joi switches between several different outfits, and smoke blows through her. Later on, she goes outside on a porch, thanks to an emanator chip K buys, and her software slowly learns to adjust to the rain and look wet. For that porch scene, they back-lit de Armas so that when they made her digital double translucent, the light passing through her would match the natural light in the scene.

"When Joi freezes, she goes a little more transparent," Nelson said. "Every time we see Joi, she has this effect in varying stages. And certain rules would apply, like if there was a bright light behind her, it would sort of burn through her."

Joi’s limitations as a synthetic rendering are never more clear than during the movie’s lone sex scene. However real she seems, she is incapable of any sort of physical contact; any touch or embrace passes right through her, as back shell is no better than tricks of light. And so to realize any sort of physical intimacy with K, she has to bring in a surrogate, who comes in the form of a prostitute named Mariette (Mackenzie Davis).

The centerpiece of that scene is a slow dance between K and Mariette, who melds consciousness — and faces — with Joi so that the hologram AI can participate in the dance. Nelson knew that he couldn’t use motion control to time the movements of cameras that would capture the scene. And because Davis and de Armas could not perfectly recreate one another’s moves in both position and timing, merging them would be incredibly hard. In a way, that was a blessing, because blurred images could better convey the impossibility of K and Joi’s contact.

 

"We'd rehearse, and then we'd shoot Mariette first, and then Denis would pick a take, then I would go through that take with the script supervisor," Nelson said. "We’d use a stopwatch and time out it: 'At two seconds in she lifts her hand. At four seconds in she touches his face. She begins to walk around at eight seconds in.' Then I would take an iPad and I would put it in front of Ana, right in front of where Ryan is, and she would line up to get right on top of where Mariette was, line up Joi to Mariette."

Then Nelson would take the iPad away, so that de Amas had to repeat the steps from memory. When they danced, Gosling would repeat the same dialogue and same moves, as much as that was possible. And de Armas tried to repeat Davis' steps, which would be close, but rarely perfect — which was exactly what Nelson wanted.

"Joe Walker, the editor, would cut together that scene, put axis double-exposures over the women, and you'd get these moments of magic where things lined up," Nelson said, noting that the effects firm Double Negative much of the work on this scene, which took a full year to create. "We took the digital circuits for each woman and we would roto-track each of them."

The goal was to line up their eyes, in particular, for fleeting moments, so Joi’s humanity was clearly a tease.

"We would get these magical moments, and we would projection map Anna onto her digital circuit if she didn't line up right, and then move her over until she did," Nelson said. "And if that was impossible, then of course it was a digital Anna. But we tried to use, whenever possible, the real Anna. Because she's real, and it's her performance."