In this class, we learnt to synchronise the mouth and facial expressions of our face model with a short speech that we selected.
First, to capture a video from internet (YouTube) in this case to use it as a reference to build up our speech lip sync, we used ‘OBS Studio’. With this programme we can record the screen and sound of our computer’s desktop and then edit the length and export in the desired format in Adobe After Effects, or Adobe Premiere Pro. The sequence that I chose shows Charles Xavier talking to Magneto in X-Men Apocalypse (Movie Scenes, 2021).
Then, we imported it to Maya as an ‘Image Plane’. It is important to have set the timeline with the same fps (frames per second) to our clip, otherwise, it will not be in sync with the audio. In my case, I exported my clip at 30 fps so set Maya with the same value. To import the audio, we right click in the timeline and import it from there. If we want to preview the clip with the audio to double check that they are in sync, we can use ‘Playblast’ feature which will show a low-resolution preview.
Once we had our reference clip set, we started to animate the mouth creating new targets in the blend shape previously created to fit the mouth shape to each sound of the speech. Then, using keyframing we set the exact movements we wanted and smooth them editing and adjusting the key frames in the ‘Graph Editor’. A useful reference to build the mouth shape depending on the sound of each syllable, I took as a reference an online image from Preston Blair Phoneme Series (Martin, 2018).
I did not create all the phonemes as not all of them were used and I actually blended 2 or 3 phonemes at the same time to create a different phoneme with this. I also animated the tongue, the eye brows, the jaw, the head rotate, and the neck (Addam’s apple). Below there is a sequence of screen captures of all the movements together and the targets that have been used.
I really enjoyed modelling and animating the facial expressions. I definitely need to improve and refine the animation part as some of the phonemes are not as polished as I would have liked but to be my first time animating a face and lip synching a speech I think it looks really good. I feel like this is an area that I would like to explore further.
References
Martin, G. C., 2018. Preston Blair phoneme series. [Online] Available at: http://www.garycmartin.com/mouth_shapes.html [Accessed 12 December 2022].
Movie Scenes, 2021. Magneto’s Final Talk With Charles Xavier | X-Men Apocalypse (2016). [Online] Available at: https://www.youtube.com/watch?v=1gZqgfiWDh4 [Accessed 28 November 2022].