JosephBrandenburgFIEA: Soliloquy of the Stars Sprint 2

This sprint, I had 2 primary tasks.

The first task was to make a first pass of the spaghettification effect.

At first, I wanted to do it in a shader in Unreal, but I found that getting the information into the shader that I needed as not going to be easy. So for no, I have decided to make a vertex animation texture solution.

So my work shifted to Blender, because I had a plan for how to achieve it with Geomertry nodes. My first attempt at creating the effect was to make a simple logarithmic spiral.

The Logarithmic Spiral approach. Not quite what I wanted...

My second attempt was to use a curve, and sample the position of the curve. I calculated the offset from the curve by projecting the position of the mesh vertex onto the tangent of the curve. But this calculation was not correct, and it was complicated. So I had to scrap it and try again.

The first attempt with curves.

After thinking on it overnight, I realized I was overlooking the obvious solution. I can compose a matrix from the sampled point on the curve, its tangent, normal, and binormal. Then all I had to do was transform the mesh points by that matrix.

The result of the matrix approach. Look how easy it is to modify the effect!

The Geometry Nodes graph for the effect.

At this point, had a really nice effect that is easy to modify and art direct.The curves can be edited to change the shape of the effect, and the radius of the effect can easily be changed. The effect can be placed on any part of the face, and moved around as much as desired. Finally, the effect can easily be layered and repeated.

The result of layering the effect.

At this point, I just had to get it into Unreal Engine.

This was a straightforward process, all I had to do was make an animated texture sequence of the effect (For now, I have decided to use an image sequence rather than a VAT, for simplicity's sake). I made a material in Blender that shows the vectors on the surface of the mesh. It packs the -1 to 1 vector space into the 0-1 color space that a texture can hold.

The vectors of the effect displacement

The packed vectors of the effect's displacem

In order to do this, I transformed the object to its UV coordinates and rendered the image sequence with an orthographic camera. In order to prevent errors in the effect at UV seams, I used Blender's compositor to inpaint the empty part of the image (extending the texture, like texture padding). Bringing it into Unreal Engine turned out to be rather tedious - I had to import the image sequence using an "Image Media Source", hook it up to a "Media Player" and then route the media player output into a Media Texture.

Once I did all that, I had to make a material in Unreal Engine to read the packed vectors and apply them to the world offset position.

The material in Unreal Engine

With the material working, it was a simple matter of making a video with the Unreal Engine sequencer... which was also a bit confusing. It turns out, in order to control the Media Player that drives the Media Texture that drives the effect, I had to make a Media Track in the sequencer, and then attach that Media Track to the Media Texture instead.

Here is the result:

My Second Task was to set up the skinning for the Mocap characters.
This was a lot more straightforward, since I had a document from the animators that explained the whole process.

Here is the result: