The Many Visual Worlds of ‘Westworld’

JW: We actually don’t do very much previs. We work really closely with our production designer and art department in the beginning stages, and we really rely on them from a world-building standpoint. Jon Carlos came up with the idea of the tower and what it was going to look like – he’s the one who found the hotel down in Los Cabos – and then we had to build the whole city. But we really start with actual locations and concept art.
Since its October 2016 debut, HBO’s Westworld, which is based on the 1973 film of the same name, has been a consistently surprising, philosophically challenging, and sometimes frustratingly opaque sci-fi series. Like its cinematic precursor, the show takes place in a futuristic, Wild-West-themed amusement park, where a group of android “hosts” allow wealthy guests to indulge their wildest fantasies. When the hosts begin to deviate from their programmers’ carefully planned scripts, things quickly begin to get out of hand. In later seasons, the series’ setting moves to the real world, where a powerful AI rules people’s lives. Westworld’s fourth season, which premiered in June of this year, sadly has turned out to be its last, as HBO cancelled the show in November.
To execute the Game, I partnered with CoSA VFX because we can’t do motion control. If the avatar has to change from being a young girl to being someone older, to being in a different outfit, to being in a different environment, that just screams a motion-control shot. We started batting ideas around and we’re like, “What if we use depth cameras and what if we do it as a point cloud?” That gave us the freedom to create a three-dimensional asset and a three-dimensional object that can transform as we’re doing it, without having to build it in Unreal or with a camera move.
So we set up a volumetric capture stage, shot our actors, and then used LiDAR scanning and point cloud for the environments. We created this interesting painterly look that gave us the freedom for camera moves and transitions. We knew we could have created it in a game engine, but we needed it to be something that wasn’t a perfectly rendered object. We wanted to make sure it wasn’t trying to be photoreal.


  
JW: It is a difficult thing. I try to not have shared shots whenever I can. Part of the challenge is that we have to do 75% of our work in California for the tax credit. So we can’t just go to whoever we want in the world. Usually, I keep my 75%, and then I save a chunk for a specialty group like ILP or Rise, and I save some for 2D fixes. At the beginning of Season 3, I took the opportunity to see who was still in California and who had filmed TV divisions at that point. And then I just kind of figured out who’s doing what. It was like, “Okay, you’re building the tower, you’re building the city, you’re doing the robots.”

Also, the opening crane shot for when we established the town of Temperance, which is the recreation of Sweetwater. I just love that – the train coming into the station, and the music starting, and getting all those happy vibes of Season 1 again. We really got the feeling of going back to Westworld again.
AWN: Last question. What are your favorite things from Season 4?
AWN: What’s the proportion of location shooting to soundstages to LED walls?
And we keep layering on top of these things each year. And it’s always funny to me how eventually, in a production meeting or a concept meeting, we can say, “OK, so it’s in the Sublime.” Thankfully now we know what that is. But each season they keep coming up with some new thing that we have to figure out. And this year it was the game that Christina narrates, which changes as it goes along. It was Host City, and it was how to do Times Square. There were so many hours, and so much blood, sweat, and tears spent trying to figure out how to execute this story idea in a way that was visually interesting and that worked for production.
For Hale’s office in Season 3, we captured an entire set and created it in Unreal and did it as a classic virtual production. We built an LED wall, put it outside the window, and had everything translate in parallax. Basically, we just picked a camera angle, went up on a scissor lift, captured a bunch of reference photos, and then got the whole place through photogrammetry and recreated it all in Unreal.
Marked by exceptional acting, visionary design, and stunning visuals, the show has received seven Primetime Emmy Awards, including the award for Outstanding Special Visual Effects. Among the recipients of the VFX award was eight-time Primetime Emmy nominee Jay Worth, who has served as the show’s visual effects supervisor since the beginning. At this year’s VIEW Conference, we spoke with Worth about Westworld’s fourth (and final) season, his long tenure with the show, and the ways in which the series, and his role, has changed over the years.


Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.
AWN: What tools do you use for visualization and how do you work  with the VFX team? Do you do any previs?
But that’s the fun thing about the show – each year we add these layers that create this really rich depth. It becomes this rich tapestry, and you feel that depth when you’re watching the show.
I tried to keep them as separated as I could, just from a television pipeline production standpoint. We had a challenge this year where one of our vendors couldn’t finish all the work because we overloaded them late in the process. And so then we had to share backgrounds and compositing and all of those other components. It made for a challenge just to make everything match.
Jay Worth: The amazing thing about Westworld is that, when I think about it, every season has had a new challenge. So in Season 1, it was, what does the robot even look like? What does the host look like? How did the map work? The map was the big technological challenge for Season 1. It was like, “Wait, you want to have this thing, and it’s real time of day, and you want to have depth, and you want to have all the things move on it, and it has to rotate on an actual piece of Styrofoam with projectors, and how does that even work?” We had to figure out all the technology for that. And then, in Season 2, it was all these story things – what’s the “Sublime,” and how does that work, and how does it open, and what’s inside the robots’ heads?
JW: We love mixing it up. We always shoot on location whenever possible, but you never get as long as you need. Our base is in LA. When we shot in Utah during the first two seasons, we only got to shoot there four days. All of the Host City material was shot in Las Cabos in about two days. We shot in New York for I think four days. So part of the challenge for VFX is filling in the gaps when we need more of something, or need to bring something back.
AWN: With your vendors, how much are they completing on their own, versus how much needs to get passed back and forth between them? And is that still a difficult thing?
AWN: Westworld displays such a cornucopia of fabulous design, it’s probably hard to choose individual elements that stand out. But what for you are some of the highlights of Season 4?
With Westworld, there’s such a collaboration among all the departments – special effects and practical effects and makeup and production design – and we all kind of know what Westworld should feel like. And it’s now gotten to the point where they know when they need to do something practically, and I know when I need to step in to help out from the visual effects side. So it’s just that collaborative effort and making sure we’re making the right decisions before we get into post.
This season, I partnered with Magnopus and we built a futuristic Times Square in less than 10 weeks for an LED volume. Times Square probably isn’t the best environment to create on an LED Volume. It has so much depth, it has this weird canyon vibe. We shoot on film, and the pipeline for shooting on film and on an LED stage with virtual production is very difficult. After a shoot, we have to develop the film, then we have to go to the lab and watch it; then, if it’s not right, we have to try and figure out what’s going to make it look different, and then go back and do it again. I ended up being very happy with the end result. There were a couple of shots when you’re super wide and you see all the creases and you see the ceiling, and we knew we were going to have to replace those. But, all the interactive light, the real performances, and the actors feeling like they’re in this environment, is invaluable.
JW: I didn’t think I was going to like building the tower, but it turned out to be this crazy thing that actually feels real. And also the young girl, Frankie (Celeste Clark) – when her face opens up, that ended up being a pretty surprising, amazing moment for all of us. Because it has all the fun callbacks – it’s got the young boy, it’s got Dolores, and there’s that moment of, “Oh, that’s where we went.” And now you realize that poor Caleb is not who we think he is.

AWN: In a show with a heavy VFX component, over the course of a season there can be something like 3,500 shots. How do you deal with handling that amount of feature film-quality visual effects in a TV world?
JW: I think, for me, it usually comes down to making the right decisions before you get started. I’ve taken over projects where someone said yes to the wrong things early on, and then you’re stuck either having to deliver a product that you don’t necessarily love, or having to spend too much money on overtime in order to get it done, or getting into really dangerous territory and risk missing an air date. What it comes down to is having a trusting relationship with your filmmakers, and making sure that what you’re saying yes to is achievable with the time and money that you have.

Jon Carlos, our production designer, figured out what Host City was going to look like, but it was up to us to execute it. And then, the Game [a 3D world that the storytellers in the show create and manipulate] was completely on us. A character would say, “She’s a young girl. No, she’s older. She’s in the country. No, she’s in the city,” and we had to figure out how to execute that in a way that’s unique to the show and that’s also going to serve the story.” And then trying to figure out how to do Times Square was a whole different beast-and-a-half.