Advertisement Jump to content
Sign in to follow this  
McJayke

Music As Variable In Games - Advice Requested

This topic is 1882 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I have been thinking a LOT about the following idea for about a year now, and when I finally brought it up to my girlfriend and brother, it got them thinking, too. I just wanted some insight on other people's thoughts on the subject, and maybe a little guidance as to where to start in making this a reality. Okay, so that's enough of a lame introduction, let me get down to the concept.

 

I used to play this game called Beat Hazard that was a twin stick shooter. The thing that made this game stand out for me was that it would use information from your music as a variable within the game. As your MP3 would play in the background, the level of the bass frequencies would be used to determine the strength of your laser, which you would fire at approaching asteroids and enemy ships. So, songs that had more bass (dance tracks, dubstep, maybe even death metal) would give you a more powerful weapon. This was shown visually, as well, in that the more bass-heavy the music was, the more bright and vivid all of the graphics were. It is one of the only games I've found that uses music as a variable, without being a music game (such as Guitar Hero and the like).

 

This gets me thinking, though, about how many ways one could use music to control different aspects of a game, and how much more diverse that game would then be. One example I keep coming to is using the tempo of a song to control the number of enemies that are generated in a world. Another idea, which to me is far more intriguing, would be to use a 3D visual of the music to generate the terrain that one was walking on. An example of this is HERE, but it isn't exactly what I'm referring to, since my idea would analyze the file first and generate a stationary landscape. But when I saw the video I have linked there, it gave me the idea that all of this is possible. I hope this makes sense, because I honestly have tried looking for information on this, and have come up empty-handed, and I honestly don't know how I would search for it. I am going OUT OF MY MIND to figure this out.

 

Just one more example that gets me thinking is THIS. Even if you could take the 3d visualization from a track, and use it to create the waves in a body of water in the background of a game, I'm sure someone could see how intriguing that would be.

 

The main idea, though, is that music has lots of characteristics, from tempo, to volume, to peak frequency, to pan, and many beyond. Games have many variables that we would use to control them, such as number of enemy sprites, speed of character, background visuals, and obviously many more. I want to get down to using music of someone's choice to control those variables. I see this as a great idea, because it takes a basic game, and gives it infinite possibilities. If you get tired of playing the same level over and over, then just change the soundtrack, and play through something potentially completely different.

 

I'm not very experienced with any specific code. I have dabbed in numerous languages, and definitely have no trouble grasping logic. I am just looking for someone to point me in the right direction, or team up with me on a project which I am willing to devote likely all of my free time to, because I think it is an amazing idea. Maybe it's been done. I would love some more examples of where it has been done, because I would love playing this type of game. Maybe let me know which languages support the sort of libraries I would need for a project like this, so I can get to work. Any help is GREATLY appreciated. Thanks.

Share this post


Link to post
Share on other sites
Advertisement

Have you completely missed Audiosurf? It's probably the best known game that uses your music library out there. Other examples include Vib Ribbon (PS/PS2), Sequence (PC), Raycatcher (PC) and The Polynomial (PC) or Every Extend Extra Extreme (360).

Share this post


Link to post
Share on other sites
Wow. Thank you all for your responses. I checked out all of the games mentioned, and I love them all, for the most part. I also came across the term"music reactive game" while looking around, which sounds like what I'm trying to do. My question, then, is...how do they do it? Is there any one language that is easier to work with to get these sort of audio analysis results? Or any one library, for that matter? I'm so glad that you can grasp what I'm talking about here, and all responses were and are appreciated.

Share this post


Link to post
Share on other sites

If you're not an overly technical person (both as a programmer and in audio), you might find such a task quite difficult. I don't know of any direct resources, but I would suggest looking for details on how to perform spectral analysis of mp3 files to get your data, as well perhaps into steganalysis / cryptanalysis for the purposes of working out what to actually do with it.

 

This page offers no insight into how to perform the analysis, but you can see how these visual representations of the result differ, and how you could potentially use the same data to your own end:

http://blowfish.be/eac/Spectral/spectral.html

Share this post


Link to post
Share on other sites
Though I am not very intensely seasoned in programming, I do love to learn and experiment, and I hav loads of free time. I also am very experienced in music and sound, so I found the article that you referred me to very familiar. And I am grasping the concepts of how I would do this on a logic level. I guess the only thing I'm needing now is a language to mess around in. Can anyone recommend one, and perhaps some audio analysis code to get me off the ground?

Share this post


Link to post
Share on other sites
My question, then, is...how do they do it? Is there any one language that is easier to work with to get these sort of audio analysis results? Or any one library, for that matter? 

 

 

Generally, the first thing games do is change the music from "time domain" (i.e. a normal wave) to "frequency domain".  Time domain is what a normal wave file is-- a mapping of changes in air pressure with time, which is what sound is.  Frequency representation is an alternate way of describing sound in terms of its frequency components.  Do some google searches on "frequency domain representation of sound", or "time domain vs frequency domain", etc.

 

There is a well-known technique for transforming a sound from time domain to frequency domain called the FFT, Fast Fourier Transform.  once the sound is in frequency domain, you can do things like look for lots of bass, areas of high-frequency etc.  You can find source code by searching online for FFT.  Usually this is done in 'C'.

 

Another (time-domain) technique these games use is called auto-correlation.  That's typically done for beat detection.   

 

Getting much more into it would take a lot more than can be done in a forum posting, but searching those terms I highlighted should give you a bunch to chew on...

 

-Brian

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!