Animate Lip Sync

This week I was working on mouth expressions for our main character, the car dealer. Our team's first idea was to move the mouth or the mustache of the character, but I kept running into problems when connecting the joints into already existing joints. After trial and error, I went to try our second idea, applying 2D animation onto the 3D model. My first approach was using photoshop, but I realized that it would take a long time drawing the face expressions using video layer. The software that I ended up using was Animate which worked smoothly.  

The first thing I did was import my audio clip onto the timeline. By creating a new layer and naming it audio. By clicking on the audio layer click on files, import, import to stage and then selecting my audio of choice. The audio should show up on the timeline.  

Then I created a new layer called mouth and drew a default mouth shape. Then converting it into a symbol by selecting the default mouth and right mouth clicking. You will then name the symbol as you like and make sure the type is set to graphic and then click on done. Converting it to a symbol will allow me to reuse animated keyframes.  


On the right side of the screen on the properties panels you will be on the object tab. On the tab all the way down you will see a tab called looping. From there you will press on to play a single frame. This will make the animation stop on a single frame.  


By clicking on lip syncing, you will have to create mouth shapes based on the sounds.  

Next double click on the mouth symbol and start drawing out the rest of the mouth shapes by creating new keyframes. A helpful tip is to use the onion skin so you can see the outline of your previous drawing. You want to be sure that the top of the mouth does not move, and the only movement should be in the bottom for a smooth transition.  

Once all the new mouth shapes are done you will then exit the mouth symbol. Then click on the mouth symbol and in the properties panel click on lip syncing. You will then click on the thumbnail and select the matching mouth shape.

You will then select the audio layer you would like to sync with. Once clicking done you will see a bunch of keyframes on the mouth layers where adobe animate will try its best to match the audio and match it with the right mouth shape. 


Comments

Post a Comment

Popular posts from this blog

Part 1 Using Oculus lip-sync in Unreal Engine 5.3

Image
     In this week’s blog, with the help of one of my group mates Joaquin figure d out how to install the plug-in into unreal . We couldn’t figure out exactly why the plug-in would not install but Joaquin saw another OVRlipsync folder. He dropped the folder into Unreal’s plug-in folder. In U nreal   the OVRlipsync plug-in should show up in the plug- i ns folder called oculus lipsync . Once I found the plug in, I checked on the box and a window popped up to ask to restart the software for the pl ug-in to show up into unreal to be able to use it . Once that's all done I was able to use the oculus lip sync.        Starting off in the model BP in the component I added a OVRLipsync Playback Actor and audio .        Back to the content drawer in unreal I added a new folder and renamed it audio . In the folder I dragged and dropped a wav audio file. Hovering over the audio and right mouse click , I cl...
Read more

Updating the mouth

Image
     Last week I was working with the lip sync for the character . I added a new mouth for the character that looked a little bit cleaner and more simplistic. I also added the new audio for the character that we recorded.        First, I used photoshop and drew out the different mouth expressions. I used photoshop because I am a bit more comfortable and experienced using that software instead of adobe animate. I transferred the mouth expressions to adobe animate where I used the lip sync. From there I exported it out as a Png image sequence and imported it into unreal. I d id the same process where I used the media track in the sequencer to play out the lip sync.          I did run into a problem where the plane had a black background instead of being transparent. Usually when exporting a png it should be transparent. What I think went wrong in photoshop I painted the background black.      This was luckil...
Read more

Cleaning up the mouth animation in Adobe Animate

Image
       This week I was working on the final lip sync. Instead of doing clips of the audio for each phrase the main character says. We thought it would be easier if we put together all the audio we are using in one audio clip. For the process of the lip sync, I used adobe animate because it has lip sync. Where it automatically lip syncs the audio for me. As easy as it sounds, the auto lip sync isn’t 100 percent accurate. I would have to go over it a couple times making sure that the mouth doesn’t look too chatty, using unnecessary mouth movements, and making sure it aligns with the audio.      These are some things that I did for the lip sync in adobe animate when cleaning it up.       To change the mouth expression. I clicked on the keyframe that I want the mouth expression to change , clicked on the mouth that's on the scene, and on the right I clicked on the frame picker. A panel pops up with all the mouth animation that I added i...
Read more