I am doing research for implementing a high-quality lip-sync plugin in a 3D CG application for animation with audio (thus, not posting in the Game forums). So far, the APIs/SDKs encountered have been monetarily prohibitive for a single developer like myself and in these times (see Annosoft for $9500/year as a prime example). If there is one that is less costly supporting Windows and Mac in C++ for much less, links would be appreciated (haven't found one yet). Foregoing that if I must, then it'll be time to implement one myself. My background isn't strictly as a CG programmer and have no formal schooling, so it wasn't something that came up in my past (which would have been long before it was normal course material). This will be a cold start in developing a 3D lip-sync system. I'm looking for beefy information on quality lip-sync algorithms - not 'Joe's quick and dirty method'. My IEEE and ACM memberships/subscriptions have lapsed (again, thanks to the ever optimistic financial situation of late) so those resources have been barred except individual purchase of papers where possible. One book that is being considered is "Graphics Programming Methods" by Jeff Lander (who has a tutorial on lip-sync here iirc). "Stop Staring" by Jason Osipa is also under consideration but I haven't been able to determine if this is a programmers book or a users book yet (leaning towards 'users'). Basically, as part of the research stage, I am trying to gather as much useful, quality information as possible in various forms (papers, books, code, websites, etc.). It is a very complex field and the term 'lip-sync' intersects with many other non-related fields (lip-sync on DVDs/TVs, in other media formats, cartooning, and so on). Please, could anyone with previous experience please provide golden nuggets of where to find the best information? Thanks

You're asking for a lot of information on an extremely narrow subject. I would be very surprised is anyone responded with the information you are looking for. The reason they are charging 10k a year is probably because it is both a complicated technology and a niche market.

I understand your call to see if there were any who had done it before, but I doubt it, so I'd simply do some serious googling ("lip sync algorithm"), and also search Usenet.

Heres 5 seconds of googling:

http://www.idiom.com/~zilla/Papers/lipsync91/lipsync91.pdf
http://hotlab.tel.fer.hr/papers/sp06.pdf
http://www.visagetechnologies.com/products_sdk_lipsync.html
http://www3.interscience.wiley.com/journal/113447555/abstract?CRETRY=1&SRETRY=0
http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V18-4K1G450-5&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_version=1&_urlVersion=0&_userid=10&md5=cce1d71fef02ac7c51818421bdf78c7c

Already 'seriously Googled'. Started with '3D lipsync algorithm' and went from there. I've also already gathered all of those papers. :)

I find that interesting on a game development site. Lip-sync is widely used in gaming (Half-Life 2 etc.), some gaming boxes have custom lip-sync parts to their APIs, MS SAPI is mentioned quite frequently (but mine will need to be Windows and MacOS so that's out), and it is used extensively in 3D CG movies/animations (when not using facial motion capture specifically).

You are correct; it is complicated technology. Thus the reason to seek as much information as possible. But before I go about 'purchasing' every paper out there that is remotely related to this subject, it'd be better to narrow it down just a bit (having been an IEEE/ACM member, I know the numbers of papers on any given topic - they could number in the hundreds or thousands!).

Thanks,