We also had to create a topographical map to put all those pieces together, like where the bar is in relationship to the beach house, and where the garage is in relationship to all that. That was all based on Sidney and his team’s previs animation of the fight. And once they had made a red carpet of their action, we worked with them to pack it into a piece of terrain. Which is made fun by the fact that the bar and the garage are exactly the same terrain because all they did was remove the garage and put a bar down. So, we redressed a few pieces here and there and shuffled the ground around.
AWN: One of the biggest challenges with CG characters, especially lead characters that you’re going to see close-up, is getting the face and the eyes right. With all the advances in performance capture, is that still a really difficult thing for you to do?
But She-Hulk can’t do a large powerful clap – her hands are too small. So, her Hulk clap was going to have to be something different. The way it was prevised was that she claps really fast, and it creates a sort of sonic wave. She makes these sharp little claps on top of each other, and this vibrating wave of energy goes out. So, we prevised it that way and then Dan and his team figured out what that would look like. It’s this distortion-like effect radiating out from her hands that affects everything, but not as concussively as Hulk’s.
What’s also interesting is that our work on She-Hulk influenced what we did with the Hulk. Other than the color of her skin, nothing about She-Hulk’s face is different than any human being. There was no caricature. We had to get her right. When we realized how far we were going with She-Hulk, we tried to do the same thing with the Hulk. In Infinity War, and especially in End Game, the teams that worked on it did an incredible job establishing Smart Hulk. What we tried to do was to take that acting, facial, and body as close as we could to what Mark Ruffalo was doing – so, treating the Hulk a little bit like we treated She-Hulk.
GW: Dan Macarin and his team worked on that. We knew what Smart Hulk’s thunderclap was – it’s this concussive shockwave that goes out and throws everything backwards. So, it’s like an explosion without an explosion. That was just a matter of figuring out the timing and affecting every tree in the forest with it. We built the digital garage so that the roof was affected by this invisible wind passing by it. And then we just kicked up some dust. That was pretty straightforward.
As far as Todd Hulk, like Tatiana, Jon Bass did an amazing job. He was a really good actor and gave a fantastic performance. So, it was all a matter of making sure that we held onto it as much as possible. Todd is the bad guy. He’s a womanizer and a misogynist. So again, we started with an actor puppet, which was critical in allowing us to do the transformation from human to Hulk. Then we did a blend from the actor puppet to the character puppet. We started with Todd’s normal character and then we just Hulkified it. We gave it big muscles, we frizzed up his hair a little, we tried to make him look like Todd, but with a squared-off face and big muscles. And then we all realized, damn, he actually looks gorgeous. He turned into this underwear model; this idealized man.
GW: The biggest challenge for us in Episode 1 was that about half the work that we had to do on the entire series was in that episode. We had to deal with She-Hulk and Smart Hulk, we had to deal with the various environments, we had to deal with all the costumes, because She-Hulk goes through a few costume changes. And it’s not just a matter of building the costumes, it’s making sure that everybody that has a say on the costumes is happy with what they end up being. We had to deal with a couple of transformations, which are actually really hard to do because of Tatiana’s hair. She has very short frizzy hair and She-Hulk has very long flowing hair, and it’s not even a technical problem, it’s an aesthetic problem. What would it look like for frizzy hair to straighten down into long glamorous hair without looking like a morph? Those are a few things that we had to deal with.
One of the things you learn in facial animation is that it’s not hard to make a character look like a person, but it’s hard to make them look like a person while they’re acting. We discovered on previous shows is that it takes a bit of effort and a bit of time, but you can make your render look exactly like the person. But the second they start talking, you start moving your shapes around, you start creating expressions. Let’s say that person has 120 expressions that they go through to create all their dialogue. Instead of it being one frame that you’re making look like that person, all 120 frames have to look like that person. So now your problem has become 120 times harder. And to make it even worse, as you go from one frame shape to another, the translation between those two shapes is not linear, it’s curvilinear. If you don’t get all the in-betweens right, you instantly don’t see that person anymore. Your brain may buy that it’s a real person, it just won’t buy that it’s the actor.
SK-K: I did some keyframe work on a sequence with Rocket in the first Guardians of the Galaxy. I did that with a camera shooting my reference. And a supervisor who was recruiting for a big company asked me which part of the sequence wasn’t mo-capped. And I said, “None of it is mo-capped.” And he was like, “Oh, come on. Which part is it?” And I said, “No, no, it’s not mo-capped. It’s keyframe.”
Guy Williams: Smart Hulk and She-Hulk were very similar. The main difference was that we received an ingest for Smart Hulk from Avengers, whereas, for She-Hulk, we had to work with Digital Domain to come up with a creature from scratch. Even for Smart Hulk, we ran it through our pipe a little bit to validate the ingest and make sure that we were happy with how it fit in.
GW: Skaar is an interesting character who only exists in one shot. He was supposed to have a few more shots and we never knew the exact scope of the work until we got the final cut. So, we built him up to probably a higher level than we needed to. In essence, he’s Hulk as a teenager. So, he’s burly, but not the Greek god that Hulk is. He’s from the planet Sakaar, so he wears Sakaarian clothes, he has a Sakaarian hairstyle, he has some Sakaarian tattoos. It was fun to figure out what the Sakaarian version of a tattoo looks like. As far as performance, because he is straight up a teenager, we leaned into the whole “yeah, whatever” attitude. He gets introduced and he rolls his eyes and everybody’s like, “Jesus, do I have to do this?”
GW: The audience has definitely changed. They’re a lot less forgiving of bad animation or bad render or bad results.
AWN: Let’s talk about some of the main sequences. In Episode 1, which takes place on the remote island where Bruce Banner lives, we see Jennifer Walters become She-Hulk and start to come to grips with what that means. Then there’s this cousinly competition with Smart Hulk that leads to a full-on brawl and destroys a lot of real estate. What were the big challenges on that, and what does it mean to have destruction-ready assets?
Tatiana Maslany’s performance was amazing, and it’s our fault if we don’t capture what Tatiana did. We will have created a new person’s performance by not hitting her performance. So, for us, there’s a huge amount of effort that goes into being faithful to everything that’s successful in a performance.
SK-K: I would say that it’s still the core of the business for us. You want to get the people who are watching to believe that the emotions we are trying to convey are real, that they are coming from a real person. Episode 1 was the biggest focus for that for us. We had to establish that She-Hulk is going to be a convincing character when she’s fighting, when she’s smiling, when she’s living. We are chasing the appeal of the character. And the technology is great when it pushes us towards that, but we always have to remind ourselves that this is an art form. We are not trying to make something mechanically right. Just like CG in the very early days, where you felt like something was moving right, yet you didn’t relate to it because something was missing. This is still the fight that CG has.
GW: The whole episodic versus film debate has changed dramatically in five years. If you asked this question five years ago, we would have said, “Outside of Game of Thrones, episodic gets paid a quarter as much as film.” You get smaller budgets. You have to try to figure out how to stretch your dollar. It used to be that serious work was done in film and then there was some interesting work done in episodic. But Game of Thrones and a few other shows around that time started to change that, and then the pandemic solidified it.
Ultimately, what we are trying to do is get to the point where you won’t be able to tell what is what. A lot of the fight scenes are fully keyframed, because, with superheroes, none of us can go as fast or hit the pose that these guys are hitting. So, I usually get the team to use the mocap as a good first step for blocking. It gives us the timing of the actress, the intention of her movement, and the overall spacing that we need to have. But when it comes to adjusting the posing for the interpretation of what She-Hulk is, we have to put those things in. Mocap is just a fancy tool that helps us go a little faster. Especially on those really subtle acting moments, when you want to honor the performance of the actress, having the mocap to help drive that is really a formidable tool.
AWN: A lot has been made of the fact that this is the first Marvel show with a full CG lead. What materials were you given to start with? Were there any models? Did you have any concerns about being responsible for such a massive amount of up-close CG hero character animation?
On top of that is that as much as it looks like we’re down in Central America or South America, the beach house was shot on a stage. The bar and the parking garage were shot on the same outdoor lot, they just redressed it to be the garage one day and the bar the other – with about 15-20 palm trees. We had to extend all the environments out and make them look like they were in the same outdoor environment. We had to build a whole bunch of palm trees, plants, ferns, ground, to make sure that whenever we look in any direction it feels like we’re in the same place. Since we’re on the beach, there were a lot more shots than originally conceived… establishing shots, flying up and down the waterline. So, we were supposed to see a lot more of the water – not just seven or eight shots – and instead of doing some very expensive water work, we wanted plates. They were supposed to fly down to Costa Rica, find their beach, and then do a drone, or a helicopter shoot, for a week.
We played with the idea of Popeye arms or something like that, but it all looked contrived. Then we found an image of this weightlifter from the Middle East, who you could tell isn’t trying to compete in bodybuilding competitions. He competes in world’s strongest man competitions. So, he’s trying to work certain muscles up to be able to compete in certain events. He’s got massive shoulders and upper arms and thighs, but he doesn’t have a very strong waist, and his forearms aren’t that strong. He has a strong stomach, but it’s not sculpted. So, he has this almost silly appearance. We ended up making Todd like that, and it clicked.
AWN: Basically, you create a puppet from your actor and then, when that’s right, you bring the actual actors into it.
When we talk about how the art has evolved, Sid’s and my understanding of what performance is has evolved over time. All his key animators – the facial modeling team, the facial animation team – all of us have sat looking at performances, looping back and forth trying to figure out why we feel compelled to empathize with a person in this split second in time. Because if you don’t capture that little bit of magic sauce or X factor, then what happens is you just don’t care for the performance as much. The way that technology helps with that is that as your understanding grows and as you understand how to get to the answer better, you start to write your tools to help you get there faster in a more automated way. It’s not like the tool is doing any amazing work, it’s empowering the animator to get what the animator can see is the right answer.
Created by Stan Lee and John Buscema more than 40 years ago, She-Hulk first appeared in comic book form in February 1980, and the character has made regular appearances in various forms ever since. In her latest incarnation in Marvel Studios’ She-Hulk: Attorney at Law, the eponymous character (played by Emmy Award-winning actress Tatiana Maslany) is not only a green 6-foot-7-inch superpowered Hulk, but also Jennifer Walters, an attorney specializing in superhuman-oriented legal cases. Not surprisingly, her life, which would be challenging enough simply by virtue of her being a single, 30-something professional, has a few extra layers of complication.
GW: Yeah. All our official systems are based on the FACS shapes., and there’s two ways you can get FACS wrong. You can either be firing the wrong FACS shape, or you can be firing the correct FACS shape, but the FACS shape is inaccurate. When you look at an error, you’re trying to sniff out if it’s error A or error B, which is very hard to do. With an actor puppet, you can say, “I know I’m hitting the right result of the shape because I can see it on the actor’s face. I’m getting the right shape to the FACS shape.” The question then becomes, “Am I firing the right FACS shape?” The actor puppet allows you to trim down the process so that you can know where you’re making mistakes, and then you know what to fix. It becomes a ground truth. If your actor puppet looks like the plate, then you know that you’re doing the right thing. Then it just becomes a matter of interpreting that performance onto the digital character.
We haven’t seen him much in his everyday life before. And that’s what was different. We needed to make sure that the Hulk seemed like somebody who could be your neighbor. It was the part of the challenge that was fun, making the two of them slightly caricatured, yet absolutely realistic, so that it felt like they could be part of the same world.
SK-K: Probably the time element. We approached She-Hulk just like it was a feature. We didn’t cut any corners. Fortunately, with the experience that we have doing these shows, we were able to improve our pipeline, and made it as efficient as we possibly could. And I really feel that She-Hulk is probably the most amazing character I’ve done, and the show is one of the highest-quality in term of animation. In term of rendering, I was in awe. I was really surprised how far we could take it.
Fortunately, the artists entrusted with this work were members of the intrepid VFX and animation teams led by Senior VFX Supervisor Guy Williams and Senior Animation Supervisor Sidney Kombo-Kintombo at New Zealand-based Wētā FX. We spoke with Williams and Kombo-Kintombo about their heroic work in the Hulkiverse, including, among many other things, what you do when the villain you created turns out to be too hot.
AWN: Sidney, what was the most challenging thing for you and the animators on the show?
As for the destruction, I would love to say that we did some amazing new tech there, but the truth is it’s what you do whenever you blow up a tree or knock over a bush. Because we do a lot of interaction with foliage, our foliage-building tools are destruction friendly. Like when you build a fern, it gives you all the curves that go through all the joints of the fern, which you can give to the effects team. They can simulate those curves and then wrap the fern back to the curves. We get a lot of that stuff for free. For the most part, all you had to do was choreograph what you wanted to see, and then build a couple of bespoke pieces. Obviously, when the trees get knocked over, we don’t just run a splinter engine on the scene, we actually make a bespoke model so that we can get something that’s a little bit more natural and organic.
When we’re talking about something like making skin look real, we can talk about a path tracer, we can talk about the fact that it’s a frequency-based render, so it deals with color and absorption correctly. All the technological advantages that make our renders look more real. But when you’re talking about performance, it’s not technology that drives the answer, it’s the artistry of the people working on it. It’s the entire team looking at a performance and saying that looks like Tatiana, or that looks real, or it doesn’t.
AWN: You’ve worked on a ton of films and a lot of episodic shows. The volume of work that you produce on a series is generally much larger than what you do on a feature. But there’s also a different timeline and a different budget. How different is it for you to work on episodic as compared to film?
We started with models and scans. We got cyberscans of both actors, by which I mean Clear Angle photogrammetry scans. We got photoshoot reference from the scans, but on a dedicated photo shoot. We got the standard package that you would get if you were doing a digital double. One of the things that we do, especially with a creature that complex, is we create what we call an actor puppet. So, we had one for Mark Ruffalo and one for Tatiana Maslany. We do that so when we’re doing the facial animation, we make sure that we’re getting the facial shapes right and then we transfer those facial shapes onto the character puppet.
So, it just means that we have to be a lot faster on set. We have to make decisions about the way we approach things so that the shoot day never slows down for visual effects. And, on top of that, they’re making decisions to speed up their shoot day, so it’s like, “We’re not going to go to this country or to this beach to shoot, so we need you to do more in visual effects.” So, there’s a lot more negotiation about how you navigate the shoot and how it impacts the visual effects and the art department and every other department that’s involved. Even though visual effects budgets are climbing, and the work is getting to be much more interesting and much more sophisticated, you still are trying to shoot a lot more material for one project.
AWN: Your work on the character animation starts from the captured performance. We just published a behind-the-scenes featurette about the suits being used and the petabytes of data that are captured in the performance. But you’re still doing a tremendous amount of key-frame animation, right?
Visual effects haven’t changed that much. If I work on a film, I do a thousand shots. If I work on a TV show, I do a thousand shots. It’s roughly the same, and the budgets aren’t too far apart. But shooting them is still different. If you’re shooting a film, you’re trying to get 120 minutes’ worth of material on tape so that you can cut it together. If you’re shooting a series that has 10 episodes, at an hour an episode, you’re trying to get 600 minutes of film – and you can’t take five times as long to shoot it. So, there’s still a huge push for economy in the shoot day. And that does affect the visual effects, in that visual effects slow down shooting. However, if you’re doing your job as good as possible, that difference is negligible. You can’t really tell that we’re there.
AWN: How did you create the thunderclap effect – a classic Hulk move – that they both start using in the middle of their fight.
Sidney Kombo-Kintombo: I wouldn’t say “concern” – I would say it was more of a fun challenge to bring that character to life. Unlike other characters we’ve worked on, it’s a female lead, and she’s not dark. She’s not always just intense. I remember talking to some of the facial team and one of their reactions was, “Yeah, we want to work on this because we can make a character that smiles.” In these things [we normally work on], we are usually so serious. So, it was interesting in that regard.
There are as many people watching episodic shows now as going to see films in theaters. There are a lot of really good film projects that are being done for streaming services. There are a lot of good series that are being done for streaming services. The differences between the two are starting to erode. You’re starting to see TV shows that are being done at film budgets and with film timelines. They are still less lucrative just because of the simple fact that there is more work, and nobody wants to spend huge, huge money on a single project. So, there is still some economy there, but the growth disparities that made it very hard before have gone away.
But another COVID wave hit around that time, so we actually went down to Dunedin, in South Island, New Zealand, and found a beach that worked for our purposes there. It had some really cool cliffs for when she falls off the clifftop. It was very cold… it was winter. A little bit of grading and nobody knows how cold it is. We were able to make it look like beautiful tropical beaches.
We were all in love with him, and then we showed him to the studio, and they were like, “We love it too, but you can’t love him so much. He has to be a little bit more goofy or villainous somehow.” So, we worked really hard to figure out how to bulk somebody up without his being idyllic. Some of the stuff was easy. Todd has slight bags under his eyes, so we played those way up. Todd has blemishes on his skin – just like every other living creature on the planet – so we played those way up. Todd has somewhat unkempt hair, so, when he goes to Hulk form, we really unkempt his hair. But the body still was challenging, because when you put massive muscles on somebody, it looks good. It’s one of the things we identify as attractive.
Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.
AWN: Can you talk a little about how you created the characters of Skaar and Todd Hulk?
And speaking of complication (not to mention questionable segues), it probably goes without saying that handling the animation and VFX for the nine-episode series wasn’t exactly a walk in the park either. In addition to having to create She-Hulk’s character from scratch, the teams were faced with wrangling three other quasi-Hulks (Smart Hulk, Todd Hulk and Skaar); producing multiple “destruction-ready” CG environments and set extensions for Hulk Island, which contained over 30,000 trees and more than 1 million shrubs and plants; and bringing to virtual life the classic, but rarely seen, Hulk “thunderclap” effect (and its little sister version for She-Hulk), which was developed using shock waves from explosions and destruction from tropical storms for reference.
