Sign in to follow this  

Lip sync in games

This topic is 4355 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

does anyone know what route the developers of say, hl2 use to do the lip sync on their characters. ie sync the sound buffer to the facial animation realtime ingame? I presume, because all the voice audio is pre-recorded, all the sync is done offline, and then a script is used for making it match up ingame?? do they automate this process? and if so, how?

Share this post


Link to post
Share on other sites
VALVe examined some old papers regarding human speech in relation to facial movement by a Navy doctor who studied this, and it breaks down sound into something like 50 facial pre-defined positions.

Modelers grab the head of Alyx, or any character, and they define each one of those 50 positions (lets call them P1 to P50).

Then a utility reads the audio the actress recorded and creates a text script, something like:
00:00:00.2 - P34
00:00:01.5 - P12
00:00:02.7 - P02
00:00:04.9 - P33
etc...

Then the engine, in-game, plays the audio, and reads the audio-facial script and morphs that character's face into the various 50 pre-defined positions.

Perhaps I over symplified some things, but that is pretty much the gist of it.

Share this post


Link to post
Share on other sites
yeah, that's what i've got going at the moment, there are around 27 phonene's, including facial expressions.

i'm trying to find a way of doing this offline, ie. on a wav file which will output the script, and also in realtime.

any ideas on if any games currently do it that way?


Share this post


Link to post
Share on other sites

This topic is 4355 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this