More lessons from Peter B. Parker

Working with a variable pose rate (ie. a mixture of 1s, 2s, and 3s) is a huge advantage both aesthetically and technically, but it presents some problems. Issues with simulation and motion blur don’t apply to the work I’m doing right now, but issues with camera movement remain.

Consider this shot, the first to go into production on a new project I’m doing with Chris Perry.

Strobe city. The character has been animated with a variable pose rate, but the camera is on 1s. The camera kind of has to be--the aesthetic qualities of on-2s character movement let the audiences eye fill in the blanks, but moving over a static environment will just look weird and stuttery.*  The problem with this is that any time the character’s movement aligns with the camera’s, it’s going to strobe terribly as the camera gets ahead of the character on the frames where the character’s pose is held.

There are a few naive solutions, but they come with their own problems. You could just inbetween down to 1s, but then you lose all the advantages of a variable pose rate. You could parent the character to the camera, but then you’d lose the ability for the character to interact effectively with stationary objects, and every modification to the camera movement would change the character’s relationship to the environment. The character grasping the pole in this shot would have required counter-animation, and nobody wants that!

Luckily, Sony developed a much better solution for Spider-verse, and I had the chance to see the solution explained at the Spider-verse panel this SIGGRAPH. Basically, they could “stick” poses to the camera, so that the held frames pick up the camera’s movement, without changing the poses themselves.

Ever since I’ve been thinking about how to implement a similar tool myself, and this shot was a perfect test case. It actually turned out to be surprisingly simple:

What’s going on here is that the script is moving through each frame in the frame range, and as it goes it keeps a record of the camera’s matrix on the last pose it encountered (poses are defined either by the keys on the optional “keyNode” or the keys on whatever’s selected). On frames between poses, it updates the position of the “poseTransform”--a single transform set up to offset the character’s geometry without affecting the rig--using the difference between the camera’s matrix on the last pose and the camera’s current matrix, effectively dragging the pose with the camera. Then, when a new pose appears it returns the poseTransform node to its default location, ready to be dragged again. If the camera or poses are adjusted, you can simply rerun the script.

As in other code snippets on this blog, I’m importing PyMEL as “pa” rather then “pm.” which is a little unusual but became such a tradition at Anzovin Studio that I’ve stuck with it. This tool would run faster if I was using om2, but for something that’s already really fast it didn’t seem worth it.

Looked at from an objective point of view, the result ends up looking like this:

But looked at from the camera’s point of view, it looks like this:

It does a pretty amazing job getting rid of the strobe issues. I’m only running it up to frame 188, so that her hand doesn’t slip around too much on the pole once she grasps it. However, that does include the section of the shot between frame 160 and frame 188, where she grabs onto the pole but is still moving upward. Interestingly, even though her hand does slip around on the pole during this section due to the camera movement, it’s not very evident to the eye. They discovered this on Spider-verse too--they’ve got shots of Miles walking down the street on 2s while the camera tracks with him on 1s, and as long as the poses “stick” and his feet aren’t central to the shot it works fine, even though his feet are slipping all over the place. If the motion and character design are stylized and graphic in the right way, your eye just forgives these kinds of issues, the way it’s always forgiven them in drawn animation. That’s very different from the assumptions that underlie conventional CG.

It looks like they use a few variations of this approach in Spider-verse. In some shots, for instance, it looks like they are sticking the pose’s translation to screen space but allowing its rotation to continue to diverge from the camera, which seems particularly useful in cases where the camera rotates around the character. This would be pretty easy to implement if I end up needing it, but for now it looks like this script, simple though it is, should effectively solve the strobing issues I’m likely to encounter on this project.

*I’m actually considering the idea that 24fps may be insufficient for some camera moves--the idea of a camera on 48fps while characters have a variable pose rate that varies between 48 and 6 fps is something I’d like to experiment with.