Jump to content
  • Advertisement
Sign in to follow this  
Lopez

Lip sync in games

This topic is 4600 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

does anyone know what route the developers of say, hl2 use to do the lip sync on their characters. ie sync the sound buffer to the facial animation realtime ingame? I presume, because all the voice audio is pre-recorded, all the sync is done offline, and then a script is used for making it match up ingame?? do they automate this process? and if so, how?

Share this post


Link to post
Share on other sites
Advertisement
VALVe examined some old papers regarding human speech in relation to facial movement by a Navy doctor who studied this, and it breaks down sound into something like 50 facial pre-defined positions.

Modelers grab the head of Alyx, or any character, and they define each one of those 50 positions (lets call them P1 to P50).

Then a utility reads the audio the actress recorded and creates a text script, something like:
00:00:00.2 - P34
00:00:01.5 - P12
00:00:02.7 - P02
00:00:04.9 - P33
etc...

Then the engine, in-game, plays the audio, and reads the audio-facial script and morphs that character's face into the various 50 pre-defined positions.

Perhaps I over symplified some things, but that is pretty much the gist of it.

Share this post


Link to post
Share on other sites
yeah, that's what i've got going at the moment, there are around 27 phonene's, including facial expressions.

i'm trying to find a way of doing this offline, ie. on a wav file which will output the script, and also in realtime.

any ideas on if any games currently do it that way?


Share this post


Link to post
Share on other sites
i mean phoneme's, and do any games currently do it in realtime for multiplayer games, say, VIOP?

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!