Jump to content
  • Advertisement
Sign in to follow this  
Hamaya

Unity Converting audio files into numerical outputs to influence game spawns

This topic is 2260 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

For my game I need to convert audio imputs into numerical outputs. I will then be defining each numerical output so that eventually each portion that the audio imput has effects the end result creating extremely dynamic gameplay.

Basically, what I need is source code of some sort, that will allow me to sample audio in lets say somewhere around 8 or 16 channels and then take those numerical outputs and send them to something else. I have no idea where to start on this and would appreciate any leads. Looked around for a few hours and came up with a couple things, but they only helped me collect some general knowledge of what this is.

From my understanding and equalizer converts audio into a numerical output on each channels and then turns it into a visual output in bar graph form. I'd just like to stop it before the visual part. I am however, not the code monkey. This is a one man dev team and I am in over my head. I've managed to figure out the rest of the project and mostly have it figured out, but this part is just too complex for me. (I'm going to go back and learn from scratch after this...But I need this project's code finalized for a workshop that is coming up)

Language can be anything from C, to Objective C++. Any help is greatly appreciated.

I will answer any questions to the best of my ability, if it helps at all im using Unity for this project and will eventually be releasing to android and possibly IOS

Share this post


Link to post
Share on other sites
Advertisement
No one has replied because no one really knows what you are trying to do.
Sounds are nothing but numerical values in the world of computing, so to what are you trying to convert them?
Of you can’t explain it with words, explain it with a picture.

Since you did not really give any details I will make assumptions.

Why not just read the numerical values out of the OpenAL buffer objects that you are creating? (Assumption #1: You are using OpenAL.)

Why not just read the numerical values out of the .WAV file? (Assumption #2: You are using wave files.)


Even if assumption #2 is wrong, there is no reason why you could not manually open a .WAV file and parse it yourself to get the values out of the file, even if the actual sound you are playing is the same sound but in .MP3 format.


L. Spiro

Share this post


Link to post
Share on other sites

From my understanding and equalizer converts audio into a numerical output on each channels and then turns it into a visual output in bar graph form. I'd just like to stop it before the visual part. I am however, not the code monkey.


If you are trying to extract frequency information, get ready to read up on signal processing and fast fourier transform (FFT). Here is a related SO question: http://stackoverflow...using-fftw-in-c

You may be able to find a library for doing the dirty work, I don't really know of any, except if you were using XNA (doesn't sound like you are) there is MediaPlayer.GetVisualizationData http://msdn.microsof...zationdata.aspx

Share this post


Link to post
Share on other sites

Why not just read the numerical values out of the .WAV file? (Assumption #2: You are using wave files.)


Even if assumption #2 is wrong, there is no reason why you could not manually open a .WAV file and parse it yourself to get the values out of the file, even if the actual sound you are playing is the same sound but in .MP3 format.


L. Spiro


I apologize, reading over this I don't think posting this while intoxicated was a good idea. Assumption 2 is sort of correct. I will be taking MP3 files and in a similar manner to audiosurf. Reading the MP3 (or other audio codec) and translating it into numerical outputs (laztrezort said it) using FFT or something similar. The problem is, I am not sure how to code something that will change an assortment of audio codecs into numerical outputs. Even if I have to use a seperate code for each and detect what the file type is, direct which code runs I need to be able to translate audio into numbers. Frequency, BPM, Pitch, Essentially take all the elements of an equalizer and measure them throughout the song taking samples every X milliseconds. For this project, since its on a mobile OS I am experimenting with unity. Unfortunatly, I have no idea what I am doing when it comes to the whole audio bit.

EDIT: Goal: User must be able to upload his or her own music and the code needs to accept that in one way or another in several common music formats and translate it into numbers one way or another. Edited by Hamaya

Share this post


Link to post
Share on other sites
I was going to suggest using ogg to avoid any licensing complications from mp3, but if you want your users to load their own mp3s you'll need a solution. Easiest may be to see if android / unity has an inbuilt library for converting mp3 to raw samples, that is fully licensed.

Essentially you want to use a third party library to convert from mp3 / ogg to raw data, probably 16 bit stereo or stereo floats. Then as said you could use something like FFT (either a library or find some source code), or you could simply do something like add up every 'window' of 1000 samples or find the peak over a window and use this loudness to determine your stuff happening in the game.

The more info you want to get out of the audio data, the more CPU it's going to use, which may be an issue targeting mobile phones. Things like extracting BPM and pitch are not trivial, but I don't think you really need to do that.

There have been games in the past that do the exact thing, I have a hazy memory of a spaceship shooter where the tunnel you are flying through is determined by the music playing, so it's very doable.

Share this post


Link to post
Share on other sites

The more info you want to get out of the audio data, the more CPU it's going to use, which may be an issue targeting mobile phones. Things like extracting BPM and pitch are not trivial, but I don't think you really need to do that.

There have been games in the past that do the exact thing, I have a hazy memory of a spaceship shooter where the tunnel you are flying through is determined by the music playing, so it's very doable.


The reason BPM was in there was that it would directly change the speed of the mobs spawned, making it any easy way to gauge the abstract difficulty of the song. For me at least, it is a rather vital gameplay component. And, I would be generating the entirety of the level before sending the user into it, which in theory should just give me a half a minute of load time, if optimized correctly. And then the spawns should be rather trivial as they are all predetermined at that point. What do you sugggest looking into to find a source or library?

Share this post


Link to post
Share on other sites
If that's to me I can only suggest google I'm afraid.

Never done it myself. Closest I've done is autocorrelation for finding pitch, which could conceivably work in some circumstances for bpm, but I wouldn't like to rely on it. I think it could be difficult for some types of music.

http://stackoverflow.com/questions/657073/how-to-detect-bpm-of-the-song-by-programming

Some bright spark on there points out that mp3 is already stored in the frequency domain. So if you are clever you could use the mp3 data directly rather than doing your own FFT.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!