Review: SmartMouth for Flash Makes Lip-Syncing Simple

I’m neither an artist nor an animator, and yet I was able to lip-sync an audio track in next to no time, all thanks to SmartMouth by Justin Putney. This Flash Professional extension really impressed me with how quickly it was able to automate an otherwise extremely tedious task. Read on to see how it can make animating your cutscenes so much easier.

First Impressions

SmartMouth comes in a standard MXP package, like most Flash Professional extensions, so it was a snap to install: I just double-clicked the MXP and followed the on-screen instructions. (It’d be the same for any version of Flash from CS3 upwards, though for CS3 itself you must have installed the Extension Manager.)

Once I installed it and restarted Flash, I could access the main panel via Commands | Lip Sync with SmartMouth:

The Help document can be brought up by clicking the question mark button; this does a great job of explaining the separate elements of the panel, but — call me biased — I felt it could also have used a brief tutorial walking me through how to use the tool. Still, there’s a detailed tutorial over on the Adobe Developer Connection, and the process is pretty simple anyway:

First, I imported a sound track (I picked this public domain reading of one of Aesop’s Fables, from LibriVox), put it on its own layer, and set its Sync to Stream.

Next, I created a new layer for the mouth to go on. Like I said, I’m not an artist, but fortunately we have a free Lip Sync Assets pack in the Activetuts+ archives, so I downloaded and imported that. I dragged and dropped each symbol onto a frame in my MouthShapes layer.

Then I re-opened the SmartMouth panel; it had taken a guess at the layers I wanted to use for audio and animation, so all I had to do was choose the shapes that corresponded to each phoneme:

As you can see, it picked a Start Frame and an End Frame for me, so all I had to do was click Tell me, SmartMouth. This kicks off the audio analyzer, which plays the whole audio track through (visualizing it as it goes):

After that, there’s a brief wait while it adds the keyframes for each mouth sound. I picked a 45 second sound file, which took up about a thousand frames (at 24fps), and SmartMouth figured out which mouth sounds went where — and actually placed the keyframes — within twenty seconds:

All I had to do then was remove the original mouth shapes from the timeline, and add a “grin” to the end (okay, technically that last one was optional). Check out the results for yourself:

I’m impressed!

Room for Improvement

I did come across a couple of bugs while using SmartMouth. When I entered my registration key, the “Success” dialog got stuck in a loop, and kept reappearing no matter how many times I hit OK. Then, later, I tried deleting all the mouth frames which SmartMouth had placed and running it again; this made it run a lot slower, and in fact it took longer than the 60 second time limit Flash imposes, making it crash without finishing its job.

Still, neither bug was a big problem, since SmartMouth has a kind of “emergency exit”: right-click the main panel and click EXIT, and it’ll shut down, putting you back in control. Plus, if the audio is too long, you can work in chunks of a few hundred frames at a time by changing the Start and End Frame options.

I mentioned, the Help docs are well-written — but I would have liked to see tool tips on the various buttons within the panel. It’s not immediately obvious what the buttons next to End Frame are for, nor what Mode or Limit To actually do, without reading up on them. Even “Tell me, SmartMouth” doesn’t suggest a command that will automatically place symbols in the timeline. But these are just nitpicks; once you’ve used the options, you’ll know what they do.

My one major gripe was that, even though I placed the mouth shape symbols in different places around the stage, SmartMouth aligned them all when syncing to the audio (I think the mouth shape for the letter O is out of place in the SWF demo above). However, this proved to be my mistake: if I’d created a new symbol on the MouthShapes layer, and placed the individual mouth symbols inside that symbol, SmartMouth would have preserved my positionings.

My Verdict

After Ian finished Animating the Envato Community Podcast, he told me that a tool like SmartMouth would have saved him a lot of time and tedium. (Actually, he used rather more excited terms than that.) I can see why.

In that video, there were several different people talking in turn, so there were different mouths that needed to be animated. SmartMouth doesn’t have an interface for doing this specifically, but it would be pretty simple to use it for that. Either:

• separate the speakers’ voices into separate tracks on separate layers and run SmartMouth once per track,
• use the Start and End Frame boxes to isolate the section of the track corresponding to one character at a time, or
• run it once for each character and simply delete the frames that don’t match the character who’s talking.

Although SmarthMouth’s most instantly impressive feature is its ability to put the mouth symbols on the stage in sync with the vocal track, this isn’t strictly necessary. If you prefer, you can tell SmartMouth just to create a new layer with labels corresponding to each phoneme in the vocals, so you can put the graphics in manually without having to keep scrubbing the timeline to see what sound you’re supposed to be imitating. This would be useful for frame-by-frame animation, or a scene with a lot of motion.

It’s also possible to make SmartMouth export the phoneme data to an XML file; this could then be used in another platform, like Unity, or even loaded into a SWF with AS3 so that you could animate a custom avatar’s mouth dynamically. (From what I hear, Justin is working on a version of the tool specifically for that purpose.)

Overall, I highly recommend SmartMouth if you need to do any lip-syncing in Flash. The basic functionality it amazing by itself, and the extra features push it over the edge.

SmartMouth is available for purchase at the Ajar Productions website; prices start at $49.99 for a single seat, with discounts if multiple seats are bought at once.