Soliloquy of the Stars Sprint 3

This sprint I made a VAT exporter in Blender so that I wouldn't have to se the frame sequence method that I used last sprint. I have done this before, so I figured it would only take a few hours to do. However, there was a big difference with my requirements that I hadn't realized.

The result of my VAT exporter. Notice how the data is laid out in distinct ribbons.

My 3D model that I am using to do the tests is very hi-poly, 236000 vertices. I am using a hi-poly mesh because it has to have enough topology to do the extreme deformations needed by the effect. The problem is that, using a standard VAT workflow, it would result in a texture that is too wide for Unreal engine to use - because usually, VAT textures have one column of pixels per vertex, and one row of pixels per frame of animation. I didn't want to even try to deal with a texture that was 236000 pixels wide. So, instead, I made my own packed VAT that works a different way.

If the 236000-pixel wide texture is like a ribbon, then, like a ribbon, it can be cut into strips and rearranged into the shape of a square. That is what my work was, this week - building the program to create the UV Map for reading the VAT, and to pull the data from the scene and make the VAT itself.

The second challenge came when I decided to use NumPy, because using plain Python would be annoyingly slow. I wanted to do this to gain some experience in NumPy, because it is an essential tool for high-performance programming. But programming with NumPy is challenging, because, instead of writing code that loops through individual vertices and places the vertex coordinate in individual color channels of individual pixels in an image, NumPy code operates on whole arrays of data all at once. So instead of dealing with things one at a time, my code used an array of vertex indices to generate an array of indices to pick pixels in a flat array of pixel byte data. I had to use slices to separate the X, Y, and Z vertex coordinates into the R, G, and B components of the pixels.

The most difficult part of writing this program was that it was hard to debug. You can't do meaningful print debugging with 236,000 values. printing only a few values makes it hard to notice trends. The breakthrough occurred when I realized I could just put the information I was looking for into the texture itself, in place of the VAT information. Sure enough, I was running into the classic programming problem - an off-by-one error. 

Working with NumPy, however, means that I can build new instances of the VFX and iterate on them very easily. I think I will also be able to read the texture with Niagara particles.

Off-By-One and worse, flipped on one axis.

You might ask me why I chose to do this in Blender instead of using a mesh cache to load it into Houdini and use Houdini's VAT export. There are a few reasons... 1) because I know what my code is doing and I can reverse-engineer it to unpack it in a shader more easily; 2) because it is important for me to practice with NumPy; 3) because I didn't realize it would be so difficult.

Ultimately, the real problem wasn't that this was a difficult problem - it's that I have been overwhelmed with other work, sleep deprived, afflicted with seasonal allergies, and preoccupied with events like EA day. I just wasn't able to think clearly until today. Everything was simple today when I sat down to work!

Anyhow, I exported the texture to Unreal Engine and set up the material. It's working now!

Working and driven by the Sequencer in Unreal Engine.

I also spent some time working on skinning the updated character mesh, and helping Kaleigha figure out a problem with the existing character mesh. I am going to finish this work during the next sprint.
I'm also (finally) going to be able to focus entirely on the visual aspect of the VFX. Hooray!


Here are my Perforce Submissions:

In Unreal Engine

Source Assets. I'm sure it will finish eventually, right?

Soliloquy of the Stars Sprint 2

 This sprint, I had 2 primary tasks.

The first task was to make a first pass of the spaghettification effect.

At first, I wanted to do it in a shader in Unreal, but I found that getting the information into the shader that I needed as not going to be easy. So for no, I have decided to make a vertex animation texture solution.

So my work shifted to Blender, because I had a plan for how to achieve it with Geomertry nodes. My first attempt at creating the effect was to make a simple logarithmic spiral.

The Logarithmic Spiral approach. Not quite what I wanted...

 My second attempt was to use a curve, and sample the position of the curve. I calculated the offset from the curve by projecting the position of the mesh vertex onto the tangent of the curve. But this calculation was not correct, and it was complicated. So I had to scrap it and try again.

The first attempt with curves.

  After thinking on it overnight, I realized I was overlooking the obvious solution. I can compose a matrix from the sampled point on the curve, its tangent, normal, and binormal. Then all I had to do was transform the mesh points by that matrix.

The result of the matrix approach. Look how easy it is to modify the effect!

The Geometry Nodes graph for the effect.

At this point, had a really nice effect that is easy to modify and art direct.The curves can be edited to change the shape of the effect, and the radius of the effect can easily be changed. The effect can be placed on any part of the face, and moved around as much as desired. Finally, the effect can easily be layered and repeated.

The result of layering the effect.

At this point, I just had to get it into Unreal Engine.

This was a straightforward process, all I had to do was make an animated texture sequence of the effect (For now, I have decided to use an image sequence rather than a VAT, for simplicity's sake). I made a material in Blender that shows the vectors on the surface of the mesh. It packs the -1 to 1 vector space into the 0-1 color space that a texture can hold.

The vectors of the effect displacement

 

  

The packed vectors of the effect's displacem

In order to do this, I transformed the object to its UV coordinates and rendered the image sequence with an orthographic camera. In order to prevent errors in the effect at UV seams, I used Blender's compositor to inpaint the empty part of the image (extending the texture, like texture padding). Bringing it into Unreal Engine turned out to be rather tedious - I had to import the image sequence using an "Image Media Source", hook it up to a "Media Player" and then route the media player output into a Media Texture.

Once I did all that, I had to make a material in Unreal Engine to read the packed vectors and apply them to the world offset position.

The material in Unreal Engine 

With the material working, it was a simple matter of making a video with the Unreal Engine sequencer... which was also a bit confusing. It turns out, in order to control the Media Player that drives the Media Texture that drives the effect, I had to make a Media Track in the sequencer, and then attach that Media Track to the Media Texture instead.

Here is the result:

 My Second Task was to set up the skinning for the  Mocap characters.
This was a lot more straightforward, since I had a document from the animators that explained the whole process.

Here is the result:

The character in its rest pose.

And taking a pose to show the effect of skinning.

Perforce screenshots:

These screenshots show the submission for the skinning task and the spaghettification effect first pass.

Soliloquy of the Stars Sprint 1

Tech artists didn't have a whole lot to do this time around, so our main contributition as a team was to discuss the VFX assets and divy them up among our tech artists. My VFX Asset is the Spaghettification effect. We also had a team meeing, where I contributed by helping to take notes.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Above are my notes from the first meeting.

 

I spent my time this week discussing with the lead how the effect should look and gathering references on out team reference board.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

I will be building this effect for future sprints!

Artist Interviews

Here are my slides

And here is the P4 turn in location and screenshot: //CLASSES/Cohort22/Classes/Art/CommonArt/Semester_02/ArtistInterviews/BrandenburgJoseph/joseph_brandenburg_Artist Interviews.pptx

Common Art Interactive Experience A Stage

This week I created a new material for the TV screens.

I updated the skinning for the characters.

I put a lot of work into the lighting, too. This has consistently been the hardest part of the process.

Finally, I updated the heartbeat sound effect. This week was mostly about quality assurance, and fortunately there was not a lot left to do!

Common Art Interactive Experience Stage 4 - First Bake - 'B' Stage

This week, I was mostly focused on creating a polished experience. First, I added a menu, with some pre-selection animation for the menu items and a fade-in/fade-out effect.

I updated the blueprints in the game to work with controllers, and set up the Sneak animation for the killer.

Finally, I made an ending sequence after the killer hides the old man's body.

I also added a lot of sound effects - the menu sounds, the updated footstep sounds for the sneak animation, the impact sound during the murder, the murmurs of the old man, the maniacal laugh of the killer, the schizophrenic voices in his head, probably a few more. Overall, this week was a lot easier than prior weeks and all I really needed to do was look for areas to improve things. Oh, and I touched up the lighting a little. I will focus more attention on the lighting next week since the menu and sounds are now working. Here is a video of the current state of the game, make sure to turn sound ON to hear the new audio!

Week 11 - High Res Meshing - 'C' Stage

This week my responsibility was to level up the lighting and work on the audio some.

I completely re-did the lighting this week. I aimed for much more nuance, better contrast, and more distinct color zones. I think the work really paid off and gave Richard's awesome shaders a chance to shine. I even added a flicker effect to sell the dark, moody amosphere. With the sounds and effects Richard added, it really feels good.

In order to achieve this, I had to overhaul my lighting tool again - now it has the ability to choose the type of light, to place any number of lights along the cylinder, and to control all kinds of light settings. For example, the light intensity is now set with Candela units, and there is a way to control attenuation radius and shadow hardness.

In addition to the lighting, I updated the rigs to prevent the flipping that was happening in the head earlier. It turns out, Maya doesn't do the math the way I thought it did... and I have no idea why it defaults to doing it wrong! I just had to set the interpolation type on the Parent Constraint to "shortest" and it worked as expected. Most of my work on the rig is not visible, since I spent a bunch of time improving the code and updating the generator with the ability to color the controls. This was a lot of work, but it isn't visible in the engine.

Finally, I worked on adding some sound to the game: the rain sounds, footsteps, creepy horror sting, and heartbeat effect. Richard added the hum of the lights and the flickering noise.