Jump to content
  • Advertisement
Sign in to follow this  
jon723

Synchronizing graphics to music

This topic is 4589 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello, I've been trying to find articles on synchronizing the rendering of objects to music but I havent been able to find anything (so far) online. Does anyone have any experience doing this and if so how do you go about it? Jon

Share this post


Link to post
Share on other sites
Advertisement
You can just use a time based animation key's for your graphics, and as long as you start the audio at the same time, they will stay in good enough sync.

Another possibility, I think (I could be wrong), that FMod has tag points that you can insert into audio tracks that call callbacks during playback for syncronization - of course, you can do this with any audio lib with some additional coding.

Share this post


Link to post
Share on other sites
Quote:
Original post by pjcast
You can just use a time based animation key's for your graphics, and as long as you start the audio at the same time, they will stay in good enough sync.


So you mean just keep track of the time while the song is playing and at certain points (after 'x' seconds) call my different rendering routines?

Share this post


Link to post
Share on other sites
Yeah. Just use a timer and probably a file (or script) that contains what your objects should be doing at certain points in time - you would have had to of setup the file before hand of course ;)

On a side note, I thought about writting a ogg contained scriptted animation codec to be multiplexed with an audio file for just such things (think, Audio/Video stream file, only with Audio and Text Scripted streams).. Playback would be similiar to that of a movie, only it would update 3D object(s) scale/pos/animations/etc instead :) - Though, never had the time or need yet.

Share this post


Link to post
Share on other sites
I just looked at the FMOD manual and it has functions to get the BMP for sounds that are playing as well as support for a lot of different file formats. It looks like FMOD has everything I'm looking for without reinventing the wheel.

Share this post


Link to post
Share on other sites
I've done a simple script based lip sync/facial animation demo that uses an .ogg file and a cue list ascii file.

The ascii file just stores time stamps with keyframe numbers, and everything seems to work okay.

I believe this is the way Halflife 2 does it.

Share this post


Link to post
Share on other sites
Triggers and scripts are defiantly a good way to go. For instance, if you want a certain sound to play when you walk on a certain surface, this would be done via inheritance and a script file. Hard coding it will just be incredibly difficult and/or harder to maintain.

As for syncing with animation, you could embed into the animation file ‘audio tags’ or something of the like. When one of these audio tags gets ‘hit’, fade out of the currently playing effect, and fade into the new one, which is defined by the tag.

Share this post


Link to post
Share on other sites
When going with the script approach are you checking the script file every frame or do you load the script at the initial load and check whatever script structure you have as the program is running?

Jon

Share this post


Link to post
Share on other sites
I load the script file when I load the .ogg. Because it's time based, I don't have to check it every frame.

I hold the keyframe array in memory. First frame, I store the current and next keyframe, along with their time stamps, and increment the current cue. The animation is then interpolated between keyframe 1 and keyframe 2 for the duration of the animation. When the engine's timer >= keyframe 2, I just replace the 2 keyframes incrementally, so key 1 is now key 2, and key 3 is now 2. I use a SLERP for the interpolation, and have had no problems so far. I am currently trying to get it to blend smoothly between different scripts.

Hope that helps.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!