Jump to content
  • Advertisement
Sign in to follow this  
Madvillain

Visualization of audio

This topic is 4838 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I am currently working on a puzzle game with heave focus on audio and visual elements. In the past I have never had to sync video and audio in my projects and so I have little knowledge in this field. I think most visual programs that visualize audio use a combination of beat detection and the audio output levels as input, am I right about this or am I on the wrong track? I looked for articles here on Gamedev, but I didn't really find anything that seemed to fit my needs ... so plese, I would apprichiate pointers from anyone who have made programs that sync visual effects with audio or visualizes audio output. Best regards, Madvillainy.

Share this post


Link to post
Share on other sites
Advertisement
What do you actually want to do?

Visualizers, like you'll find in a media player, usually use the sound signal as input, and measure things like loudness and spectral distribution of the sound to generate control inputs to their visual algorithm. The "sync" between audio and video is usually not that great -- it depends on what your display frame rate is versus what your sound card output latency is.

However, puzzle games don't usually involve "visualizers" like this. If there's a puzzle where you're supposed to pull a specific lever only during a certain section of a song, some meta-data will be stored separately from the actual audio, saying that "lever is pullable between audio sample 135,837 and 219,884". Typically, there will be at least 50 milliseconds latency in your sound card, and only about 10-17 ms latency in the display, so you might want to compensate for that.

If this explanation doesn't help, then you need to be specific about what you actually are trying to accomplish -- what is the puzzle, and how does it relate to the audio?

Share this post


Link to post
Share on other sites
The only way the actual puzzle relates to the audio is scoring
more points brings you further along in the song, but
I have that figured out all ready. The visualization however,
which is not supposed to be more than eye-candy in the background,
I have no idea how to implement. How are these attack levels and
such measured? I am currently using FMOD ... is there a more powerfull
audio API that would suite my needs better? I have seen a lot of good
effects syncing audio with graphics in many scene demos, so I know
that it is possible.

Cheers,

Madvillainy.

Share this post


Link to post
Share on other sites
Look up the code for milkdrop, it was recently open sourced.

Basically, you'll wanna do as it was suggested abouve. I'd suggest prefetching some audio packets from your sound API, doing a fourier transform, then using the output frequency values as inputs to your graphics. Winamp has a system like this one built in that's very highly customizable, you may want to take a look at how that works. I know the source isn't available, but if you look at a lot of the equations, you can see how its dealing with the data.

if you're not a good friend of math, I'd suggest you get reacquainted. Most of what you need is somewhere in the bowls of a calculus two book. Visualizations are really a lot of fun once you get to playing with them, as it's all just different wave equations taking their ques from the wave energies of the sound.

go nuts, and post back what you come up with!

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!