Below is our complete interview with ILM VFX Supervisor John Knoll. Excerpts from this interview were previously published on April 3, 2017.
To support the home release of Rogue One: A Star Wars Story, Syfy Wire went out to the Industrial Light & Magic and Lucasfilm offices to take a look behind the scenes of the film and speak with some of the people involved.
For John Knoll, ILM VFX Supervisor, his history with the company goes back to his childhood. Off a cold call by his father when he was a 15-year-old and happened to be visiting the San Francisco area, the young Knoll got a tour of ILM, getting to look behind the scenes of the original Star Wars movies being made at the time. He was instantly bit by the bug, and less than a decade later he'd begin his visual effects career.
Now at ILM, Knoll was the Visual Effects Supervisor for the entire prequel trilogy, and after several other major franchises and VFX powerhouse films like Avatar and Pacific Rim, returned to the Star Wars franchise for Rogue One, this time as both VFX Supervisor and Executive Producer. The second title came because Rogue One was his baby -- Knoll actually pitched the story idea of telling the story hinted at in the first Star Wars opening crawl and was present for every level of development.
In this full interview, we talked to Knoll about how George Lucas, founder of ILM and Lucasfilm, still has his presence felt at his former companies, how ILM continues to innovate in the film industry, and how practical and digital effects work together to bring to life the most realistic Star Wars film yet.
John, the last time we spoke over the phone, we talked about how the Executive Producer role was a new challenge for you, in addition to being Visual Effects Supervisor. Now that you've had a taste of being involved in all those other decisions, is that something that you're looking towards doing more in the future?
John Knoll: Well, if I was in the position of doing something like that again, I would enjoy doing that, yeah.
Was it more important to you because this was a story you pitched, to be more involved?
Knoll: Yeah, I didn't specifically ask for that. It was something that when [Lucasfilm President] Kathy [Kennedy] decided to take this picture into production, she offered it. She said that I should be an exec producer on the show so that I have more of a say on how this idea gets turned into a reality.
Your team really gets to show off new technologies, and I know you're constantly working on these new technologies; is that accelerated when you're actually on a show and you run into a problem, or is it something you try to keep going all the time?
Knoll: To some extent, that's something we always try to do; it's part of the culture at ILM to never accept that the status quo is as good as it can be. Part of the process is always, 'Is there a better way?' We try to think through if there's something we can do better creatively or technically, or just is more efficient.
This dissatisfaction with the status quo and constantly playing with a better way, I think is something that George Lucas really was a major foster of. The way that Lucasfilm used ILM was George never restricted his thinking to things that he knew could be executed with the tools at the time. He would write what he thought would be cool and what he wanted from a storytelling standpoint with the assumption that, 'Well, they'll figure it out!'
That sort of throwing the gauntlet down of ‘All right, well, you guys are going to figure out how to do this,' drove a lot of real innovation at the company. How we rose to those challenges, we developed technology that we were then able to offer to other clients and other projects. It was just generally good for the industry and good for us in particular. I felt that we should continue that legacy. As Lucasfilm is developing IP and we're working on our projects, we should be using those films to advance the ball further down the field, and to make things better for the rest of the company and the rest of the industry.
So you see things like that new virtual camera being used in future projects?
Knoll: Oh, absolutely.
That space battle at the Battle of Scarif is stunning; every moment in it has something for everyone: it has a major story beat, it has an Easter egg for someone to spot, it has that incredible sweeping action. Take me through constructing that a little because I know it has to be an incredible project, working with Gareth Edwards, with your animators -- it's almost a whole movie on its own!
Knoll: Yeah, what happened was the edit took longer than we planned after we wrapped principal photography and it went into edit. Gareth's shooting style of shooting these very long takes and fishing around for angles, he'd shoot a 20 minute-long take! We wouldn't cut between takes, so we'd roll, and we'd run the action, then he'd say 'Reset' and find a different bit of coverage or a different angle. So we iterate on trying to get what he wanted, we'd see what it looks like, and he'd finally say, 'All right, I need a break.'
There's a lot of really good material that we got from that, but it's a real challenge for editorial – there aren't slates on a lot of that, and there's tons and tons of material to go through, so that was a big challenge, just to organize all of that and start cutting it.
So as we put the film together, there were things that were too long or didn't work right, and we thought we could make it work better if we reshoot this portion or that portion. We had this big pick-up shoot that we did in the summer, and the editorial group was very focused on sorting out all that live action and making it work.
So, the space battle, which was largely a virtual thing and done in post was kind of left until the end. Editorial was so busy with it that they finally said, 'You guys need to figure this out, we don't have time!' So they gave us how the story beats would cut in with the action; so the first beat is about the fleet arriving and the first wave of X-Wings that peel off to go through the shield gate. Then it's a lot of what needs to happen in each beat, and left a lot of what shots those beats are going to turn into up to us to figure out! In fact, one of the story beats was: 'and then the Rebels take out two Star Destroyers in an interesting way!' (laughs) And then they left that up to us to figure out how we'd do that in a way that was interesting and different.
The Old Republic hammerheads to the rescue!
Knoll: Yeah! That was something that I came up with, this idea of pushing one Destroyer into another, using a ship as essentially a tugboat. That was really great fun. We had very specific story beats that were meant to be communicated every time we went back to the space battle, but it was up to us how we clearly and efficiently tell those moments, and at the same time include things that are exciting and satisfying and something you haven't seen before.
Blowing something up by hitting the weak spot and making the reactor blow up has just been done so many times, even outside of Star Wars, that I really didn't want to do that. I love this idea of making it all about the mechanical damage.
Yeah, the brute force, too, is a very non-Star Wars thing, we don't usually see that being used, especially by the Rebellion. What's something you learned or developed for Rogue One that ILM will be able to take directly into other Star Wars movies?
Knoll: Something we often struggle with on pictures is the right way to shoot live-action elements that are for an environment that's very complicated from a lighting standpoint. An example is a starship flying through an environment that's constantly changing. So in the space battle, we're flying around in X-wings that are flying around, in and out of the space dock, curving around and getting the [light] bounce from the planet, having explosions and lasers going off all around. So how do you shoot that on stage?
Well before, you'd have a green light here that represents a laser from this side, and a light over there that represents a laser from that side, and this orange light that will be an explosion, and then we'll have bounce cards we'll bring in to represent how we're flying past the sunlit part of the planet; it's all approximate and not very accurate, and it's a little unsatisfying and hard to make look real.
So there's a technique we've been using for years in computer graphics to use photographed images to light objects, and what we did on this show was an experiment with giant LED screens, the same kinds that are used for billboards or jumbotrons at sporting events, and we circled around the set with those, and used those to light the set. That way, we can prepare graphics and animations of what this environment is, and those will actually light the characters and ships; so when there's an explosion going off next to that ship, there's an explosion on the LED screen, and it's animated to travel past them. You actually see that lighting change on the actors, and reflected on their helmets.
That method of using that image-based lighting technique to light actors and sets was really successful. Our innovation behind it was really about lighting, but it was helpful to the actors, too! Now they're not just looking at an empty blue screen, they're actually seeing a crude representation of what the final scene is going to be. They don't have to wonder about, 'Which direction do I look for where that ship is or where that explosion is.' It helps them get into character and deliver a more true performance.
Very cool. The other thing you showed us today during the presentation was some of that mix between practical and digital effects. Adding in practical elements into digital sets, is that mostly for lighting, or is it more about getting the performance?
Knoll: Those proxy sets? Yeah, it was a good case where I'm usually arguing, even when we can't build a full set, I usually argue that we build a foreground at least. There are a few cases, where even that, if it's only going to be on screen for a few seconds, it's hard to make the case. If you're not going to have anything, though, how are you going to light your actors in a meaningful way? So having at least the basic forms that were about the right colors seemed like it would be a good way to work. It's harder to composite because we have to resort to rotoscope instead of blue screen extraction, but the benefit is much higher; you get a higher level of realism, so that as you drop actors into that, they really think they're there. It's a technique I'm really proud of and worked really well, so that's something else we'll take into the future.
Yeah, it's interesting, most people hear ILM, and they think about the huge space battles and crazy special effects sequences, but then it's the minutia of lighting and bringing that realism …
Knoll: Yeah, a really important part of the job is integrating with the whole crew. Working with the director of photography and the production designer and everybody to try to make sure we find that appropriate mix of what we do practically and what we leave off for post, so the result is the best we can make it.