Visualization of audio
Author
I am currently working on a puzzle game with heave focus on
audio and visual elements. In the past I have never had to
sync video and audio in my projects and so I have little
knowledge in this field. I think most visual programs that
visualize audio use a combination of beat detection and
the audio output levels as input, am I right about this
or am I on the wrong track? I looked for articles here on
Gamedev, but I didn't really find anything that seemed to
fit my needs ... so plese, I would apprichiate pointers from
anyone who have made programs that sync visual effects with
audio or visualizes audio output.
Best regards,
Madvillainy.
What do you actually want to do?
Visualizers, like you'll find in a media player, usually use the sound signal as input, and measure things like loudness and spectral distribution of the sound to generate control inputs to their visual algorithm. The "sync" between audio and video is usually not that great -- it depends on what your display frame rate is versus what your sound card output latency is.
However, puzzle games don't usually involve "visualizers" like this. If there's a puzzle where you're supposed to pull a specific lever only during a certain section of a song, some meta-data will be stored separately from the actual audio, saying that "lever is pullable between audio sample 135,837 and 219,884". Typically, there will be at least 50 milliseconds latency in your sound card, and only about 10-17 ms latency in the display, so you might want to compensate for that.
If this explanation doesn't help, then you need to be specific about what you actually are trying to accomplish -- what is the puzzle, and how does it relate to the audio?
Visualizers, like you'll find in a media player, usually use the sound signal as input, and measure things like loudness and spectral distribution of the sound to generate control inputs to their visual algorithm. The "sync" between audio and video is usually not that great -- it depends on what your display frame rate is versus what your sound card output latency is.
However, puzzle games don't usually involve "visualizers" like this. If there's a puzzle where you're supposed to pull a specific lever only during a certain section of a song, some meta-data will be stored separately from the actual audio, saying that "lever is pullable between audio sample 135,837 and 219,884". Typically, there will be at least 50 milliseconds latency in your sound card, and only about 10-17 ms latency in the display, so you might want to compensate for that.
If this explanation doesn't help, then you need to be specific about what you actually are trying to accomplish -- what is the puzzle, and how does it relate to the audio?
Author
The only way the actual puzzle relates to the audio is scoring
more points brings you further along in the song, but
I have that figured out all ready. The visualization however,
which is not supposed to be more than eye-candy in the background,
I have no idea how to implement. How are these attack levels and
such measured? I am currently using FMOD ... is there a more powerfull
audio API that would suite my needs better? I have seen a lot of good
effects syncing audio with graphics in many scene demos, so I know
that it is possible.
Cheers,
Madvillainy.
more points brings you further along in the song, but
I have that figured out all ready. The visualization however,
which is not supposed to be more than eye-candy in the background,
I have no idea how to implement. How are these attack levels and
such measured? I am currently using FMOD ... is there a more powerfull
audio API that would suite my needs better? I have seen a lot of good
effects syncing audio with graphics in many scene demos, so I know
that it is possible.
Cheers,
Madvillainy.
Look up the code for milkdrop, it was recently open sourced.
Basically, you'll wanna do as it was suggested abouve. I'd suggest prefetching some audio packets from your sound API, doing a fourier transform, then using the output frequency values as inputs to your graphics. Winamp has a system like this one built in that's very highly customizable, you may want to take a look at how that works. I know the source isn't available, but if you look at a lot of the equations, you can see how its dealing with the data.
if you're not a good friend of math, I'd suggest you get reacquainted. Most of what you need is somewhere in the bowls of a calculus two book. Visualizations are really a lot of fun once you get to playing with them, as it's all just different wave equations taking their ques from the wave energies of the sound.
go nuts, and post back what you come up with!
Basically, you'll wanna do as it was suggested abouve. I'd suggest prefetching some audio packets from your sound API, doing a fourier transform, then using the output frequency values as inputs to your graphics. Winamp has a system like this one built in that's very highly customizable, you may want to take a look at how that works. I know the source isn't available, but if you look at a lot of the equations, you can see how its dealing with the data.
if you're not a good friend of math, I'd suggest you get reacquainted. Most of what you need is somewhere in the bowls of a calculus two book. Visualizations are really a lot of fun once you get to playing with them, as it's all just different wave equations taking their ques from the wave energies of the sound.
go nuts, and post back what you come up with!
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
