Interface for synchronizing sound to animation

Started by
3 comments, last by KulSeran 18 years, 3 months ago
I've been looking all over and coming up dry for interfaces I can use to synchronize sound, music and effects with animation. I know I'm looking for something like a NL video editing but I'm so clueless I'm not coming up with anything recognizable on the search engines. Got clues??
Advertisement
That's a very generic question. It's not a difficult task unless you're using someone else's sound and/or animation code. What exactly are you making, and what are you using to make it?
I've used several animation programs before for building keyframe animations. The most recent effort was Maya's PLE where I got REALLY frustrated because it matched none of the Maya books I was supposed to be learning from. At any rate there was no sound function in any of them.

On the other side of it I've used some low end sound mixers to make some effects and vocals but no hook up to animation.

I've used adobe premiere and aftereffects for home learning...my son wanted to make "real movies" then never touched the programs (grr). After effects had the really neat capability of allowing you to bring in several channels of sound time and mix them to the video tracks...but obviously wrong format for the output.

I want to develop a game where music and effects can be linked to ; interactive objects, environment location, character states and behavioral or choice triggers...and I'm not sure what all else yet.
On the animation end I will need to generate animations for standard states and interactions between multiple characters, interactive objects, environmental locations, environmental states ...and a few more things I won't think of until I'm neck deep in it.

I am in the world of speculation trying to figure out what I will need to put the various parts of my project together and get a price range for each. OK. here comes a second mortgage! AGH!

Mentally I have a picture of an interface that allows me to bring in multiple characters and objects and maybe a background, allows me to seperately load the animation files that will be timed to each other. Load all the sound effects, music and vocals. Then lets me tweak speed, minor position changes,speed berween keyframes, sound initiation,duration, speed,pitch,volume and again probably a few things I haven't thought of yet. Then output everything into files I can use in the C++ environment.

In standard sounds studios the action is rigid and you must synch the sound to it. I think I will want an interface that allows me to adjust both to each other.

There's loads I don't know yet like is midi a preferred format over mpg, do I want surround sound effects, stereo or flat etc. There is a bunch of material available on how to create sound files that stand alone. I think I need a way to develop them in conjunction with....

I don't want to generate any files except my own. Obviously it would need to import 3DSMax files and or Maya files and standard midi, mpg or wave formats.

Is there such an interface commercially available or will I need to write my own?
Apologies for being too generic. I only have a rough idea of what it will take to bring my "game vision" to life and am trying to find all the tools I need to "make it so". So far I've lucked out with free learning software that will need to be upgraded before final production and about $700 in books. Tutorials are neat but books generally go a little deeper besides you really shouldn't smoke and swill caffiene at the keyboard and the book doesn't care.
Ok, well, I don't know if what you are asking for tends to exist in game development in the form that you expect. Things like Adobe Premiere have no real equivalents in games because there is so much more going on than just synchronising animation and sound, such as artificial intelligence, networking, collision detection, drawing a GUI, reading input, etc. The nearest thing I can think of is if you write games in Flash or Director, which work on keyframes and can probably make this part of your project a lot easier. However, they are severely limited in other areas, and you already mentioned C++, so I'll leave out that avenue.

What you will typically have in a game, among other things, are 2 totally independent systems for animation and sound. If your animation system works on keyframes, it will probably have the capability to issue a callback on frames of your choosing. This is effectively a user-configurable notification of having reached a certain point. What you'd then do, is write a little code to link that callback to the sound system, playing the sound you require.

As far as I am aware, you won't find anything that will export "files I can use in the C++ environment" because there are simply too many permutations here. There's no standardised way to handle animation, and no standardised way to handle sound, nor is it even necessarily the case that everybody synchronises sound to video, or that they even use keyframes, etc.

So, I'm afraid the bad news is that you have to dig deeper into whatever code you would use to animate things, and whatever code you would use to play audio, and decide how to link them yourself.

Quote:is midi a preferred format over mpg, do I want surround sound effects, stereo or flat etc.


MIDI is not a preferred format - its main use is for communicating with external musical instruments. By MPG I assume you mean MP3, which is the audio part of MPG, and yes, that is a commonly used format but often WAV is preferred as it requires less CPU power to play back. As for sound effects, you almost always want 16 bit 44KHz mono sounds - surround and stereo effects are performed by the sound engine which will position them in 3D space for you.
I'd like to mention that API's like FMod have the ability to read in "sync" tags from some audio formats. If you find audio software that
lets you insert these tags into the music, FMod would allow you recieve triggered events at every "sync", thus once you setup the music, you can use
the sync points as a triggers to force the animation to move along at a particular speed / switch states at a particular time.

The example I remember being mentioned when i read about it, was a for timing that little bouncing ball that you see trace words in a singalong.

This incase you want the oposite of Kylotan's idea of Animation -> soundeffect sycn.
You can also do the Sound -> Animation sync

This topic is closed to new replies.

Advertisement