Announcing Ephemeral Rigging for the Unreal Engine

As mentioned in my last post, there’s a reason why I’ve been pretty silent lately. I’ve hinted at the existence of some secret project, a creation for which the world was not yet ready.

I’m very pleased to announce that, thanks to the assistance of a MegaGrant from Epic Games, Notional Pipe is working on bringing fully contextual ephemeral rigging and dense animation data editing to the Unreal Engine! Check out our first demo for ephemeral manipulation:

Yes, that’s the ephemeral system working in Control Rig! Unlike previous tools I’ve created with Unreal, this isn’t hacked together with duct tape and chewing gum–thanks to Epic’s support, Tagore and I have been able to devote the time to do this one properly.

I won’t say we haven’t experienced some pain during the process, largely due to there being very limited documentation on how Control Rig works internally, but Unreal in general (and Control Rig in particular) is a far more inviting environment for us to develop animation tools in than Maya has ever been. Unreal doesn’t force you into a dependency graph model the way Maya does. Instead, a “Rig Unit”--that’s what nodes in Control Rig are called–is allowed to do anything it wants to the entire hierarchy. Ephemeral rigging fits into this model much more smoothly.

We’re already working on expanding the system to include the kinds of dense animation data editing tools I’ve always wanted. Animating ephemerally in a fully real-time environment is going to be incredible.

Ephemeral rigging spotted in the wild!

Do you recall that bygone time, the halcyon days of about four months ago? I made a promise then, as a foolish young man. Specifically, I promised I’d keep updating this blog on a regular basis.

And then, uh, I didn’t.

In my defense, the reason why I didn’t is that super exciting things are happening! Unfortunately, I can’t tell you about any of these super exciting things yet. So instead, let's talk about some sightings of ephemeral rigging in the wild!

Both UE and Cascadeur are using forms of ephemeral rigging! In both cases we have methods of manipulating the character that are instantaneous (ie. they modify the characters pose but don’t create relationships that persist in the scene). In the case of Unreal, those are Proxy controls, while Cascaduer seems to generally operates on the principle that posing and interpolation are separate concerns. But neither of them are really the same thing as the ephemeral system that Tagore and I are building.

This has caused me to think a lot about how to identify the differences between the systems. When other systems mostly weren’t ephemeral*, it was easy to put all the concepts we’re playing with under the heading of “ephemeral rigging.” But clearly, the term “ephemeral” isn’t quite enough to describe what our ephemeral system is trying to achieve anymore, since these two systems satisfy some of the ways I’ve previously defined the term while accomplishing something very different. This also comes as we’re abandoning the term “interpolationless.” The problem with “interpolationless” is that it’s not really true–sure, we’re not doing continually recalculated interpolation using curves like most animation systems, but interpolation still occurs when you drag the breakdown slider or generate inbetweens.

So here’s what I’ve come up with: In addition to being ephemeral, our system is also contextual, stateless, and multi-pose.

It’s contextual because all rig behavior depends on context–context in this case meaning modes, options, and current selection. Unreal’s proxy controls would be an example of something that’s ephemeral but not contextual–you author the ephemeral aspects of the rig to behave in a specific, predefined manner. In our system, you don’t define any specific behavior in response to a particular interaction–instead, you define a web of possible relationships between the controls in your rig, and what actually happens depends on context.

One really important point about this is that it lets you define a lot of quite complex rig behavior very simply–there are so many possible permutations to your web of possible relationships that a single contextual rig basically “unfolds” to hundreds, or possibly thousands in extreme cases, of conventional rigs.

Our system is also stateless because there is no rig state that changes over time. Cascadeur would be an example of a system that allows for ephemeral interaction, but is not stateless–you can manipulate your character ephemerally (with the assistance of machine learning, in this case) but you are still choosing specific states of the character that affect its behavior over time (ie. setting an IK key).

One of the things that people find most difficult to get their heads around when I demo ephemeral rigging and animation is that statelessness implies that nothing is ever “set” in the rig. When setting glue groups, for instance, you do not say “these controls are glued together at this particular time”. You just glue them together, and when you’re doing something that requires them to not be glued, you turn it off. This is vastly more flexible then a system that changes state over time, as it does not lock you into any particular assumptions about what should be attached to what at what time.

And finally, being multi-pose means that interaction can occur across time. Like this:

That’s the latest version of the ephemeral system, moving multiple poses at once! And that works for any ephemeral behavior. As an example, take a look at this video of me polishing motion ephemerally:

We’re finally starting to approach the workflow I’ve always been aiming for–one where you can “sculpt” motion directly as easily as you can pose!

These attributes have some interesting interactions. For instance, a rig can’t be entirely contextual unless it’s also stateless–if you have state changing over time then that would need to override the context of your settings and selection in some way in order to be reflected in the rig’s behavior. And while multi-pose editing of dense animation data is possible with a rig that isn’t contextual or stateless, it would be pretty awkward. You really need all three to get the full benefit of the approach we’re taking.

* You can make an argument that FBIK and even Biped could be considered sort of ephemeral given this definition, as they also evaluate the rig on interaction in a way that can differ from how the rig interpolates. Defining things is hard!

A new year

Over the last year or so, you may have noticed my posts on Just to do Something Bad have dwindled to nothing but announcements of our Notional Pipe streams, with nary a long-winded post on the philosophy of rigging or inscrutable cartoon to be found.

There are a lot of reasons for that, but they basically all boil down to my life being very busy. I’m working full time for The Third Floor now (on things which are extremely cool but of which I cannot speak), Contour Rig Tools was released last year (with the attendant need for support), we’re preparing documentation for Little Bird so it can be released on the Unreal marketplace, and I’m working on proposing another SIGGRAPH talk. Also, I’m now the father of a teenager, which is, as I understand the kids these days would characterize it, a lot.

But that doesn’t mean that no work has been going into the ephemeral system! Quite the contrary, a lot has changed and I’m hoping to have a swath of new features to show soon. I’ll start out with a quick demo of the reproportion slider.

As I’ve probably pointed out a few times, the ephemeral system does not enforce proportions on any character, so it’s useful to be able to restore proportions when needed. Previously you could kinda do this by using the Zero slider with the blend restricted to translation only, but this was awkward and really only worked in Forward mode. The reproportion slider, on the other hand, works in any mode, letting you do things like restore proportions to the arm while keeping the hand in place.

You may also note that the ephemeral system is a lot faster now, and has a nifty bottom bar ui reminiscent of AnimBot. It’s fast enough now that I’m considering trying to do multi-pose interactions interactively—we’ll see how that goes.

We should have a lot more new stuff to show soon, but in the meantime, here’s a link to tomorrow’s Notional Pipe stream where we’ll be showing more development of the leg component:

Leg component stream

In our next stream I’m going to start building the leg component.

This is the real meat of the whole component building thing—it’s by far the most complex component in the system and will use all the things we’ve learned so far.

Here’s the previous one, where you can see me fumble around a bunch making a component reversible, because rigging!

Componentized rigging streams

Here’s our first two componentized rigging streams!

To be perfectly honest, the first one really doesn’t get to the point across very well—my head really wasn’t in the game when we did that one. The second one is much better and I backed up and explained some of the basic concepts again, so I actually recommend that people start with that one.

And here’s our third, which you can join at Tuesday at 5pm EST!

To do a brief explanation, componentized rigging is a way of creating a rig out of reusable components, rather then scripting an auto-rig system. Each of these components is just a Maya file that is referenced into your rig file. But critically, the component can only communicate with the rest of the rig through specific, predefined inputs and outputs.

That might sound like a strange way to rig, but it has huge advantages. Because each component depends wholly on it’s inputs to define its proportions, those proportions can be changed any any time, allowing you to reset a characters proportions without rebuilding the rig! And because each component presents a specific “interface” to the rest of the rig, you can trivially swap out different kinds of arms, for instance, without disturbing the rest of the rig at all. A componentized rigging system is also much, much easier to develop then a traditional auto-rigger.

Because Contour bones and joints make default states like “bone length” explicit, it’s extremely easy to do componentized rigging with Contour, and you can see that in the streams above if you can sit through all my digressions!

Our componentized system builds on the work of Raffaele Fragapane, who popularized the concept on Cult of Rig.

Little Bird nominated for an emmy! (And also you can watch it now!)

There are many announcements about Little Bird! To start with, we’ve been nominated for an Emmy in the Childrens and Family/Outstanding Short Form Program catagory!

https://theemmys.tv/cafe-nominations-2022/

Not only that, but we can finally post it publicly for all to see!

Thirdly, and this has been up for a bit but we neglected to tell anyway, you can get the Little Bird plugin on github now.

We’re still working on documentation, so it’s probably pretty hard to just jump into using it right now, but feel free get it and play around. Heck, it’s open source, so feel free to modify it as you see fit and make PRs for me. (Of course, it’s all blueprints, which are binary files, so it will be pretty hard for me to review those PRs and you may need to send me a bunch of explanation of what you’ve actually done, but still).

We are such stuff as streams are made on.

We had to take a few months off from streaming, but I’m happy to report that next Tuesday Notional Pipe streams will be back!

This time we’ll be discussing componentized rigging, which is a topic I’ve wanted to talk about for a while. Frequent viewers may recall that I don’t like auto-rigging systems, which add an extra layer to the rig creation process that frequently gums up the works. But you’re obviously not going to build every rig from scratch! Componentized rigging, a concept I first encountered through Raffaele Fragapane’s Cult of Rig, gets you all of the reproducibility of auto-rigging without the ugly rig build and rebuild process.

Contour works exceptionally well with componentized rigging. In fact, the rig components I’ll be building on stream are likely to be shipped with Contour in the future! I hope to see you there.

Whatever happened to the teenage stream?

Hey, it was only a month or so ago that we were doing weekly Notional Pipe streams! Then…silence.

I know there have been a lot of wild rumors, so I want to set the record straight: we absolutely did not go to space. I did go to SIGGRAPH, though, and then had about ten thousand different things to do when I got back.

Streaming will resume soon with a series on Componentized Rigging with Contour. I’ll let you all know when it happens!

An Ephemeral live stream appears from the shadows!

When all hope seemed lost, an Ephemeral live stream appeared, as if from nowhere! And when it’s job was done it vanished, as mysteriously as it had arrived.

A more prosaic explanation is that I forgot to announce the stream this morning, so I’m now announcing it 45 minutes before it actually starts. Sorry, hope you can join us anyway!

It’s too bad because this one is a great opportunity to watch me fall on my face in front of the world! Today we’ll be using a new build of the ephemeral system, hot off the compiler, with color coding for ephemeral controls. Will it fail at a critical moment? The only way to find out will be to join us at 4pm EST!

Ephemeral live stream!

It’s that time again! Join us today as we continue our adventures in ephemeral/interpolationless animation.

Last time we went over using the ephemeral system’s "Glue Groups” feature to make it easy to attach the box and the rabbit’s hands together and manipulate them in either direction:

This time we’re going to take the key poses we put down and fill them out. Maybe if I rant less this time we’ll get farther!

Hope to see you there at 4pm EST.

How to polish animation with an ephemeral rig

Here’s Tuesday’s stream:

I want to call attention to how much more time I spent conceptualizing the motion via key poses then I did polishing and inbetweening. Taking a shot from blocking to final using the ephemeral approach can be extraordinarily quick! This keeps the animators focus where it belongs, on the big questions (like acting and appeal) not the little ones that don’t actually matter (like why is my elbow bobbling around on frame 23?).

I get a lot of questions around how polish works in an ephemeral/interpolationless approach. The idea of polishing motion when your motion is just a dense stream of poses, with no curves or graph editor, is a pretty alien one to lots of animators! Reflexively, people often assume is must be more difficult, and that you pay for ease of posing with difficulty of polish.

Nothing could be further from the truth! In fact, I’ve found polish to be far easier. I used to have to go through a lengthy process of splining and cleaning up curves—now I just fix any problem I see whenever it comes up, directly and without any fuss. It’s a lot more like a digital painter or zBrush sculptor, who can just go in with a brush and correct any little issues without disturbing the rest of the piece. The ephemeral system’s multi-pose editing tools make that easy, and those tools are still evolving.

I’ve put together an excerpt with just some of the polish bits from the stream:

The ephemeral system can process motion in ways conventional tools cannot,. In this case, for instance, the “smooth over time” tool can “filter” the motion by looking at the entire pose or any section of it, not just a specific anim curve.

Ephemeral animation demo part 2

Part 1 of our ephemeral animation demo went very well!

It also went pretty slowly, as there was a lot to discuss. We got some key poses down and talked a lot about pose-based animation technique and how to use the ephemeral system. Today, we’ll be filling out the shot with more poses and probably getting to some inbetweening. You’ll get to see me finish a section of the shot, without ever opening a graph editor!

Join us at 4pm EST at https://youtu.be/i_O6fsN2vkU

Live ephemeral animation demo

We won’t be doing a stream today due to Memorial Day, but our next stream is so important I’m announcing it early! On 4pm EST Tuesday June 7th I’ll begin an ephemeral animation demo. I’m going to animate that alien rabbit on stream, in real time, showing exactly how I’d approach animating a shot with the ephemeral system! Come join us at https://youtu.be/N9maUnlqy9E!

I get a lot of questions like “how can you actually animate a whole shots this way?” This stream should answer those questions!

And here’s the last stream, showing the setup for the rig I’ll be using for the animation demo:

This one wasn’t our most approachable stream—the ephemeral system doesn’t yet have any UI for setting up rigs, just animating them, so you see me using a lot of Python and without much visual feedback on what the system is actually doing. But animating with the ephemeral system is quite intuitive, and rigging should be soon too—we just have to turn our attention to UI. You’ll get to see what animating with it is like on the 7th!

Ephemeral streams

Today, we’re going to start a series of streams on ephemeral rigging and animation! We’re going to start by taking our weird alien rabbit and rigging it ephemerally. Then in future streams we’ll animate it in a shot live!

Come check it out at https://youtu.be/c7f2aaam-OY

Plus, here’s the recording of our previous stream on the Little Bird rendering and compositing system:

Little Bird premiers at Shortnite!

I’m happy to announce that Little Bird is premiering at Shortnite!

That’s a short film festival happening in Fortnite! As a venue, Fortnite has many advantages. If you go to Annecy, can you dress up like Batman and shoot people? Yes, but you’d probably get arrested. So that’s a point in Shortnite’s favor, I think.

And for anyone who couldn’t make it, here’s our discussion of Little Bird facial rigging and animating a zero-g character with an ephemeral rig!

I forgot to mention it on stream, but the Aliyah face geometry we examine was based on a sculpt by Carol Cornils. Go check out the short for the rest of the credits!

Next stream we’ll take a break from rigging and take a look at the other Little Bird—the real time multipass rendering and compositing plugin for Unreal we developed to create the short.