Categories
A journey through my master
Tag: Maya
In this class, we set up the lightning and colour corrected our model to get it ready to render and compositing later on After Effects.
To create the illumination of my scene, I wanted to recreate the real lighting of the scene I took as reference. I added 3 spotlights to my scene: one is in front of the model to illuminate the face, and the other two are behind as backlights.
Since the lip sync looked a bit odd without a tongue, I modelled one, added a standard surface material with some shiny highlights (to give a wet look) and animated it in the parts with ‘L’, ‘D’, ‘S’, and ‘T’ sounds.
Using the ‘Hypershade’ editor, I added two colour correct nodes: one linked to the base colour and coat of the skin material, and the second linked to the specular colour of the skin material. I added a soft yellowish base colour to the skin as is shown in the reference clip but then added a blue/purple highlight with the specular colour to make it a bit more interesting. I also added a subsurface scattering to make the skin a bit more translucent, so it looks more realistic.
After I was happy with my lighting, I set my project render details to half resolution to have a relatively quick render to review the final look. Thankfully, I did not have to change anything as I like how it ended so I set my project at HD 1920×1080 resolution and render the final lip sync animation as an alpha.
The final render turned out to be good and it is ready for compositing with a background and some nice VFX. So I imported it to After Effects and searched for a proper background for the scene. I found this already blurred background with a futuristic look so I added it to the comp and colour corrected it to make it darker, more saturated and showing more blue hues (I added colour balance and brightness and contrast effects).
Then I also colour corrected the face as it seemed a little flat so using the ‘Levels’ effect increased the contrast and added a bit of more shadows and saturation.
Finally, I also scaled up the face and I scaled down the background towards to end to make the scene more dynamic and simulate the camera movement.
I think that the final scene turned out good, as I like the contrast between the blue/purple background, and the yellowish skin colour of the face with the subtle blue/purple highlights on the head and forehead. Overall, I think the scene looks realistic and the lip sync and head movements looks pretty natural.
References
topntp26, freepik. Blurred abstract background interior view looking out toward to empty office lobby and entrance doors and glass curtain wall with frame. [Online] Available at: Available at: https://www.freepik.com/free-photo/blurred-abstract-background-interior-view-looking-out-toward-empty-office-lobby-entrance-doors-glass-curtain-wall-with-frame_1254627.htm#query=building%20indoor&position=36&from_view=search&track=sph [Accessed 12 December 2022]
Categories
Week 9: Speech Lip Sync in Maya
In this class, we learnt to synchronise the mouth and facial expressions of our face model with a short speech that we selected.
First, to capture a video from internet (YouTube) in this case to use it as a reference to build up our speech lip sync, we used ‘OBS Studio’. With this programme we can record the screen and sound of our computer’s desktop and then edit the length and export in the desired format in Adobe After Effects, or Adobe Premiere Pro. The sequence that I chose shows Charles Xavier talking to Magneto in X-Men Apocalypse (Movie Scenes, 2021).
Then, we imported it to Maya as an ‘Image Plane’. It is important to have set the timeline with the same fps (frames per second) to our clip, otherwise, it will not be in sync with the audio. In my case, I exported my clip at 30 fps so set Maya with the same value. To import the audio, we right click in the timeline and import it from there. If we want to preview the clip with the audio to double check that they are in sync, we can use ‘Playblast’ feature which will show a low-resolution preview.
Once we had our reference clip set, we started to animate the mouth creating new targets in the blend shape previously created to fit the mouth shape to each sound of the speech. Then, using keyframing we set the exact movements we wanted and smooth them editing and adjusting the key frames in the ‘Graph Editor’. A useful reference to build the mouth shape depending on the sound of each syllable, I took as a reference an online image from Preston Blair Phoneme Series (Martin, 2018).
I did not create all the phonemes as not all of them were used and I actually blended 2 or 3 phonemes at the same time to create a different phoneme with this. I also animated the tongue, the eye brows, the jaw, the head rotate, and the neck (Addam’s apple). Below there is a sequence of screen captures of all the movements together and the targets that have been used.
I really enjoyed modelling and animating the facial expressions. I definitely need to improve and refine the animation part as some of the phonemes are not as polished as I would have liked but to be my first time animating a face and lip synching a speech I think it looks really good. I feel like this is an area that I would like to explore further.
References
Martin, G. C., 2018. Preston Blair phoneme series. [Online] Available at: http://www.garycmartin.com/mouth_shapes.html [Accessed 12 December 2022].
Movie Scenes, 2021. Magneto’s Final Talk With Charles Xavier | X-Men Apocalypse (2016). [Online] Available at: https://www.youtube.com/watch?v=1gZqgfiWDh4 [Accessed 28 November 2022].
In this week’s lecture, we learnt how to edit blend shapes to create facial expressions and how to make those actions interact with each other so we achieve a natural movement when animating.
Creating blend shapes and targets inside these, we can animate our face’s expressions. One by one I created the expressions as full smile, half smile, frown, half frown, blinking eyes, and open mouth. I built up the expressions using the brush and relax tools. I also added a correction in the mouth for when it is opened to relax the sides of it and make it look more realistic. Since this correction is only needed when the mouth opens, I set the mouth correction target as a driven key and the jaw joint as driver and key framed the animation.
Initially, I struggle a bit creating the facial expressions as sometimes I forgot to switch the ‘Edit’ button of the target I was manipulating at the time and I had some errors when animating that affected previous targets. Also, I seemed to have a duplicated head node in the background and some other duplicated ‘set’ nodes that were giving me an error message all the time. After asking Nick for help, he found this duplicated nodes in the ‘Node Editor’, and after delete them, my face model started to work better. I had to delete all the targets and the blend shape I had and start from scratch. I did this numerous times before finding the real error, and it was really time consuming, but at the end of the day, it was a really good practice and I learnt how to solve the error myself for future projects.
In this session, we reminded how to create blend shapes to animate the facial expressions and how to create a ‘rig’ or ‘skeleton’ to animated the head and the mouth of our model.
Using the ‘Shape Editor’ tool, we can create a blend shape or shape variation in order to set the facial expressions of our model. On each blend shape, we need to add ‘target’ points with which we could create our movements or reshapes, for example the eyes opening and closing, the mouth smile, or the eye brows frown.
In order to animate the head and the mouth opening, we created a ‘rig’ or ‘skeleton’ that will determine the joints of the neck and jaw. After setting the rig, we bound the skin of the model to our rig and painted the skin weights to add the influence parts of our model (the parts the will be more influence by the rig movement).
Lastly, we created the model’s set of teeth and added them to the rig influence.
I struggled a bit with the painting of the weights to open the mouth of the model. I had to adjust my mesh with the ‘soft brush’ and ‘relax’ tools so it started to respond appropriately.
This week, we learnt how to add a UV texture to an organic model, a human face in this case, using both Maya and Mudbox.
In Maya, we imported the skin texture to the project. Then we created a UV map from the model in the ‘UV Editor’, and using the ‘grab’ tool, we started to adjust the UV map to the texture imported. Since the texture imported was designed for models with opened eyes (ours had the eyes closed), we exported it to Mudbox and, using the stamp tool (similar tool to Photoshop stamp), we edited the texture to match the closed eyes of our model. Once finished, we imported the edited texture back to Maya and re-adjusted it. Since the texture was looking completely flat, we added a bump map using the ‘Hypershade’ to add the pores, marks, and facial lines effect to the skin. Finally, we also added a UV map texture to the eyes and, using the ‘Animation editor’ we opened the eyelids of the model so we could see the eyes’ texture.
I had some issues with the UV map as my model mesh needed to be adjusted in the middle part of the nose (I had some triangulated mesh there so needed to make it squared and follow the rows and columns of squares, to make it more symmetric). Once adjusted, the UV map started to respond better and I could adjust the skin texture more accurately.
In this lecture, we discovered the basics of organic modelling using Quad Draw modelling workflow to created a Head model.
First, we downloaded a head model from wiki.polycount.com to use it as reference. Then in Maya, we imported our head and set it up to make it a ‘sticky’ surface. Once this feature is live, we used the Quad Draw tool to create a mask with the only half of the shape of the head. Then we refined the eyes, mouth, nostrils, ears and rest of the head traits. Once the half of the face was fully set up, we used ‘Duplicate Special’ to create the other side of the face. We also created the eye balls and mirrored one of them so any changes made in one side it is reflected in the other side too (also used topology feature to change some of the face features).
The following sequence of screenshots shows my process to develop my 3D head model:
The most challenging part of the model were the ears as their are pretty irregular, however, I think I achieved a good result at the end. I also refined the face and gave it more angular features so the model had a more characteristic look.
This week we finished up our air balloon adding UV textures.
We learnt how to set up a UV map with the ‘UV Editor’ from an object so the texture looks realistic when added. Later on, we downloaded some textures and added them the the UV map created on each part of the balloon. We also changed their appearance playing around with ‘colour’, ‘specular’, ‘transmission’, ‘geometry’, etc. As these images look too flat and no too realistic, we created a ‘bump map’ using the ‘hypershade’ editor to add more detail to the texture. In the ‘hypershade’ editor we also saw how to join different nods to create different texture effects. Lastly, we created a short animation of the model, spinning the air balloon 360 degrees and showing only the object and not the background, and then rendered it in EXR. This would be used in our Nuke class as alpha matte to make a composite with it.
Since I am interested in focusing on modelling and texturing in Maya, this was a good practice to improve my knowledge on this. I look forward to explore more organic modelling and all its possibilities.
In this lesson, we learnt further modelling tools more focused on starting our model with a drawn line that will evolve into a 3D shape with revolve tool, some extrusion, and some UV and texturing. We also modelled an air balloon that we will use next week in our Nuke’s session.
We started modelling a simple line using the EP Curve tool. This shape had to be kind of a slice of the final object we were going to model, in this first case a wine glass and a wine bottle. After our curve shape was drawn, we used ‘Revolve’ to transform it into a 3D shape. In addition, we reshaped our 3D object repositioning the control points of the curve shape until we were happy with out design. We also learnt how to create a glass effect adding a glass material and finished the scene adding lighting and choosing a background.
Using the same EP Curve tool, we continued to build up our air balloon. Then we played with extrude, duplicate special, deform (bend tool to warp the object), and used the UV editor to add texture to the model (used cloth, brushed metal, and rubber textures).
I really enjoyed this assignment as I realised I love 3D modelling in Maya so I would love to further my knowledge in this aspect.
In this session, we went through the basic modelling tools of Maya in order to shape a Lego Man.
We started shaping the head out of a cylinder primitive, and using subdivisions and the bevel tool to model the top and bottom features of the Lego Man. Then we continued with cube primitives. Using subdivisions, extrude, edge loops, and transform and position tools, we managed to shape them into the main body and legs.The hip of the Lego Man was a challenge since we started with a cylinder which was flattened and then picking the top vertex of this, we extruded them both sides and shaped them into a rectangle. The arms were made out of a cylinder and then shaped the shoulder part with the sculpting tools to pull and soften the mesh. The hands were sculpted using a cylinder with a centre circle and cut in half, then shaped it adding edge loops and transform and position tools.
Afterward, I wanted to colour it so I added a new material (‘Standard Surface’) to each part of the Lego Man and picked the colours.
In the face of the Lego Man, I wanted to add their smily face. I thought of adding a picture of the smily face to the mesh but since I wasn’t totally sure how to do this, I researched a tutorial on YouTube. It was actually simpler than I thought, I just had to add a new material – ‘Lambert’, and then in ‘colour’ select to add a ‘file’ to it. So I downloaded a Lego smiley face from internet and adjusted it to the Lego Man’s face.
And lastly, using a ‘Sky Dome Light’, I added a nice background, lighting and a cast shadow with a plane as I learnt last week and this is my final result:
A few weeks later, I also made a short animation of this model to make a nice presentation to be added in my term 1 showreel.