Creative Capture

To those untrained in its usage or mechanics, technology can seem akin to magic.  People expect it to "just work."  No technology bears this label more than motion capture.

Thanks to high profile films like Avatar, the use of the technology is better known to the public at large.  However most people's understanding is simplistic as: Actor performs in a silly outfit, data goes into the computer, character appears on screen performing those actions.  Most people don't realize motion capture is merely a single step in a long series to bring digital characters to life.

Billy Rivers is an Animator for Epic Games.  Though his official title is Animator, he feels a more accurate description of his duties would be Motion Capture Animator.  He is responsible for running the set on capture days, processing and cleaning all the data produced, and finally handing that data off to the animation team.


Why is a motion capture (“mocap”) animator important for the success of a game like Gears 3?

As a mocap animator, specifically, I'm feeding the rest of the pipeline for everything but creature work.  If it was a biped and it moved in Gears of War it was probably at one point touched by mocap.  Whether or not it was prototyped on the mocap stage to make it really quick or it was a nearly finalized animation with secondary motion, the process wouldn't be as fast as it is without mocap.

There's just no way a room full of animators could produce 90 minutes worth of cinematics plus all the custom gameplay animations per character in the time we needed them.  We produce so much that it's impossible to do without mocap to speed the process along.


Tell me about your role at Epic.

My title is Animator.  I would say I'm more of a motion capture animator which seems kind of silly, splitting hairs really.  I focus much more on the motion capture pipeline and processing side of things.  I'm on the floor running the shoots making sure everything is working properly.  I then take and clean the data making sure the weights look right, the hips are positioned correctly, things like that.

I take the output of that process and distribute it to the animation team who then adds further detail like secondary and facial animation.  I also take the data and create gameplay animation from it.  I'll take it from the first stage of working with the actors and markers and raw data all the way up to importing it into the game.  Putting weapons in characters’ hands and adding secondary motion.  I touch everything that happens with biped characters.

I would definitely call myself a motion capture animator but with a strong emphasis on "animator" because I do so much creature work as well.  I animated a lot of the leviathan in Gears 3 and some of the polyp attacks.

I imagine to most people when they hear “motion capture” they believe it to be a simple “plug and play” process:  the actor does their thing and it goes straight into the game or movie.  There's just so much extra work that takes place between capture and what you see on screen.

Exactly, and we don't capture anything as granular as the face or hands.  We have a lot of skilled animators on the team.  A number of them, for all of Gears 3, focused on cinematic facial and hand animations.  They made sure the characters were looking at each other and really communicating their lines.  Some of these animators have such amazing skill sets and backgrounds.  It's awesome seeing the stuff they come up with.

The characters’ emotions were not based solely off the actors’ body motions.  It was really sold by the animators’ touch on that mocap afterwards.  It was really cool to see how the raw mocap I gave them evolved.  They were able to add so much more character and depth to the performances.  I get to do that same sort of fun stuff with the creatures and characters in different scenes as well.

A lot of this work was done previously by Scott Dossett.  I stepped into his shoes and was there for the majority of the cinematics work.  He did a lot of leviathan work.  When he left it was intimidating to have to fill his giant shoes.  He's a skilled animator and knows his way around the mocap stage.  It was a ton of fun learning from him and all the talented people here.

Epic really gives a lot of room to grow.  The average number of years an Epic employee has been in the industry is 10 years.  When I came in I had about two.  I was put in charge of something extremely important, the cinematics for Gears of War 3.  There was a huge weight of responsibility, but in a good way.  I felt like they gave me a lot of trust really quickly and my leads said I didn't disappoint.  It was great to see the amazing tools he (Dossett) had created and imagine how we can expand them for future games.


What is the greatest challenge you have had to overcome, either with a personal or professional project?

I stepped into this role under Scott Dossett.  The original plan was a six-month transition to train me up on all the systems before he departed.  When the time came we ended up only having two weeks.  That was definitely a "drinking from the fire hose" scenario.  I was digesting so much information so quickly.  We were in the middle of cinematic production at the time doing two mocap shoots a week.

You either pick it up and learn or you don't and thankfully I picked it up.  Luckily it only took two weeks of crazily writing down every word he said and memorizing.  It was a really hard transition to pick up and go but at the end of the day he felt comfortable leaving and the team felt comfortable with me taking over.  It all worked out in the end.  It was very, very intimidating and a huge challenge to walk in and take over the mocap animation pipeline in two weeks.

Do the Gears 3 voice actors also perform the physical roles on the mocap stage?

All the audio is recorded ahead of time on a sound stage.  For Gears 3 we got the audio processed and cut the way it was going to appear in game.  Then we brought in local talent for the motion capture.  They're a lot of fun to work with.  We would play the audio on the mocap stage to get the timing and the actors would belt out the lines and act to it. We had a lot of random ad-lib moments that we got to put in here and there.

Do you have any specific examples you can share that made it into Gears 3?

We were capturing for the character of Ashman in the level of Char.  He was the guy who would pretend he was an ash statue and take off whenever you got too close while setting off traps.  Basically pissing off the main characters the whole time.  The actors who played him also played Baird and a lot of the other quirky smartass type of characters that are so fun and prevalent in Gears.

By the time you finally reach this guy he's been so wily and quick you think he'll have something else up his sleeve but he comes down an elevator and has nowhere to go.  At the last second we were doing a take and he froze in place in front of the actor playing Marcus.  It was obvious he would have seen him just freeze in place, he was just being funny.  Everyone busted up laughing.

Every take afterward we told him to do that because it was so funny.  It ultimately went into the game.  It was such a great random moment that came from having fun on the stage.

That's great the process accounts for those moments of inspiration and ad-lib so new performances can be discovered and made better instead of having to stick to a rigid script.

I know other studios like Naughty Dog for the Uncharted series captures their audio and motion at the same time so they have tons of ad-lib moments.  A lot of great moments develop from the characters interacting with each other.  They'll rewrite the script on the fly to incorporate those new ideas.  That's why they get so much character in the interactions in Uncharted.  I love to see that.

We're flipping it a little bit in the new DLC we just released: RAAM's Shadow.  You have a little bit more character interaction.  We've had good feedback on the cinematics because we decided to do the acting first and the dialog second.  We still had scripts that the mocap actors were performing but a lot of the timing we got from them acting out the performance on stage.

The professional voice actors then used that for the timing in the recording studio.  What that did was a lot of the scenes in the script we modified on the fly because certain interactions made more sense when the actors made subtle changes during the performance.  It was cool to see it develop organically on the set.  Hopefully we'll be seeing a lot of that in the future.


What work are you most proud of? Why?

I want to say Gears of War 3.  I'm so proud of the product.  It's so fun to play and see my animation work pop up.  In the starting sequence of Nightmare you see the big creature come out and start pounding the ground.  I got to work on that guy and it's like, "there's my guy getting sucked into the vortex!"  It was some crazy idea me and Dave Nash came up with at 11:30 at night as we were trying to get this thing into dailies for the next day.  Stuff like that makes me have a real sense of pride in the finished product.

I really liked how tight the cinematics were in RAAM's Shadow.  It was a collaboration of people saying, "we liked the cinematic process in Gears 3 but how can we do it even better?"  I feel like we learned a lot from Gears 3, refined the process, and improved on what we'd done before.  I'm proud we were able to grow so much in such a short amount of time.

I always want to do that on every project in my career.  I want to learn to do something and then find a way to improve it.  We'll never stop learning even when we're 30-year veterans in the industry.


How do you see animation and motion capture evolving over the next few years?

For me as an animator the dirty secret we share is that we don't like facial motion capture because in almost every case an animator can always add something that the fidelity of the technology won't let you capture.  Take LA Noire for example.  Their level of detail in the face was so high that it made the standard mocap on the body not jump out as much.

Until we can capture that level of fidelity across the entire body including the face, I don't see mocap as being the end all solution to things.  It will definitely speed up the process but it won't be the 1:1 solution people sometimes think it is.  I would hope to see it becoming higher and higher fidelity.

I want to see it evolve into more control given to the animators.  We want to be able to tweak an eyelid with 6 different controls.  We want to be able to put so much detailed animation into the face so you believe you're looking at a real person.  An actor can do so much but an animator will sit there and look at the same shot for hours on end.  They will add every little detail that they can.

A great example is Blizzard's latest Diablo cinematic.  It is amazing.  The humans have so many nuanced facial quirks.  They have such a talented team.  It still has that CG feel but it looks so real.  I want to see every cinematic in every game reach that level and surpass it.


Are there any techniques or best practices you have learned or developed that have served you well?

I've always known I'm not the smartest person in the room.  There's always something to learn from your peers.  It doesn't matter if you're in a small startup or the biggest established studio in the world, there's always somebody who knows something that you don't.  You can always learn from those people.

Forge relationships, talk to people about what they do even if it isn't directly related to what you do.  Animation affects every discipline.  Everything is so interconnected.  By learning what other people do and how they operate you will often find ways of improving your own work and make what you do look better.  Never stop learning.

What is the title on which you are currently working?

I'm working on Fortnite which was recently announced.  It's different than what we've worked on before, it's been a lot of fun.  I'm also working on some other unannounced projects.

Tell me a little bit about your background.

After I graduated with a degree in Animation and Visual Effects from Ex'pression College I was brought onto America's Army Emeryville as an intern.  From there I transferred to take a full time role in Alabama where I was doing animation.  I eventually progressed into mocap animation through Epic Games.

What sort of projects did you work on while in Alabama?

We worked on a wide variety of different things.  They were mostly military trainers and simulators used for the purpose of testing soldiers, weapons, and technology.  I was animating anything from biped characters to quadrupeds for testing in different simulations.


What are your inspirations?

I definitely get inspiration from films but my primary source is games.  When I see games like Uncharted really take character and story to the next level I, as an animator, want to meet or surpass that level of fidelity and find new ways to do so.

When I see other companies that have done amazing things with animation, particularly with motion capture, I want to see why they did that and find ways of dissecting it and learn how to take it further.  Not only for the games I make but for the medium as a whole.  I want to see the fidelity of games increased to the point where you're playing at Avatar levels in terms of graphics.

I imagine you were pouring over the tech they used for LA Noire.

Ya, definitely.  You tend to notice things, it almost ruins games and films because you know what they did or how they did it.  You also see things like the disconnect between the high fidelity faces and low fidelity bodies.  It tends to stand out and makes you wonder how they could incorporate what they did into full body simulations so it wouldn't have that disconnect.  It makes me wonder how that technology can be taken to the next level.  I'm a big fan of what they pulled off.  The facial details are so great.


Knowing what you know now, what advice would you have given yourself when you were just starting out?

Even though my degree was in Animation and Visual Effects, my education wasn't as specialized as it could have been.  My animation and motion capture classes were to the point and brief.  I wish I was doing mods and a lot of things for years before even attending school so I would have been more versed in animation and motion capture, almost like a pre-education.

You always have to take general education courses before you pursue your major.  For animation you're learning all the time.  You start to pay more attention to the world around you.  You watch how peoples’ faces move when they speak to you.  It's all subconsciously seeping into your head.  You need to know why those things happen in animation to really sell a character’s performance.  I kind of learned that as time went on.

If I have to make a character happy I need to know why he's happy.  I didn't realize there's a much deeper character inside everything you do.  It could be a guy walking down a street but he has to have a story behind that whether it's for a split second or a 90 minute piece.  If there's not it stands out.

It's not as easy as just turning up the smile slider in the animation controls.

Right, exactly!  There's a lot of technical things in animation of course, but there's so much to be said about the character and the story behind it.  Without all those components together you can have a sound, technically great animation that doesn't go as far as you'd hope it would.

I would tell myself to pay attention to everything around you because it all leads into what you're going to do some day;  every single movement and ball bounce as a kid.  If you see a guy walking down the stairs: why is he shifting his weight so much more to one side?  When you watch three people walking down the street: why is each individual different?

I found myself creating stories for the people I saw riding the BART train to explain why their attitude seemed a certain way.  It was a fun way to entertain myself on long rides but it all comes back to how story and character help dictate more believable animation and motion.


What motivated you to work in the game industry?

Next to my passion for animation is my passion for games.  I knew I wanted to be in games since I was a little kid.  I always want to see the next big game.  Why is it popular? Why do people like it?  Why did I pick up this title and play it for 200 hours?  I'm really interested in the psychology behind that.

Games are so much fun and such a cool medium to involve you with people.  You get to make this movie that players can control and interact with however they want.  It's so cool, I love that.


What are your favorite games and why?

A lot of the games I play don't have much mocap in them but they're very fun. My favorite games are the ones that draw me in and I can't stop playing.  Games like League of Legends.  I've been playing since it was in beta.  It is so much fun to play.  I love playing against somebody else's brain.  It's like digital chess with some twitch factor involved.

I play a lot of Gears 3 online.  Horde mode is my favorite.  It's a lot of fun to play the games you make.  I just started playing, probably to my detriment, The Old Republic.  I'm hoping I don't get hopelessly sucked in but I probably will.  It definitely feels like I'm playing a Bioware title based on how story driven it is.  It's cool they've achieved that in an MMO.  I'm dividing my time between those games and Skyrim right now too.

Wow, sounds like you have a full plate!

And two kids!  I've learned that sacrificing sleep helps give you more game time.


Thank you for your time Billy!