Jump to content
  • Advertisement
Sign in to follow this  
Hamaya

Unity Converting audio files into numerical outputs to influence game spawns

This topic is 2162 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

For my game I need to convert audio imputs into numerical outputs. I will then be defining each numerical output so that eventually each portion that the audio imput has effects the end result creating extremely dynamic gameplay.

Basically, what I need is source code of some sort, that will allow me to sample audio in lets say somewhere around 8 or 16 channels and then take those numerical outputs and send them to something else. I have no idea where to start on this and would appreciate any leads. Looked around for a few hours and came up with a couple things, but they only helped me collect some general knowledge of what this is.

From my understanding and equalizer converts audio into a numerical output on each channels and then turns it into a visual output in bar graph form. I'd just like to stop it before the visual part. I am however, not the code monkey. This is a one man dev team and I am in over my head. I've managed to figure out the rest of the project and mostly have it figured out, but this part is just too complex for me. (I'm going to go back and learn from scratch after this...But I need this project's code finalized for a workshop that is coming up)

Language can be anything from C, to Objective C++. Any help is greatly appreciated.

I will answer any questions to the best of my ability, if it helps at all im using Unity for this project and will eventually be releasing to android and possibly IOS

Share this post


Link to post
Share on other sites
Advertisement
No one has replied because no one really knows what you are trying to do.
Sounds are nothing but numerical values in the world of computing, so to what are you trying to convert them?
Of you can’t explain it with words, explain it with a picture.

Since you did not really give any details I will make assumptions.

Why not just read the numerical values out of the OpenAL buffer objects that you are creating? (Assumption #1: You are using OpenAL.)

Why not just read the numerical values out of the .WAV file? (Assumption #2: You are using wave files.)


Even if assumption #2 is wrong, there is no reason why you could not manually open a .WAV file and parse it yourself to get the values out of the file, even if the actual sound you are playing is the same sound but in .MP3 format.


L. Spiro

Share this post


Link to post
Share on other sites

From my understanding and equalizer converts audio into a numerical output on each channels and then turns it into a visual output in bar graph form. I'd just like to stop it before the visual part. I am however, not the code monkey.


If you are trying to extract frequency information, get ready to read up on signal processing and fast fourier transform (FFT). Here is a related SO question: http://stackoverflow...using-fftw-in-c

You may be able to find a library for doing the dirty work, I don't really know of any, except if you were using XNA (doesn't sound like you are) there is MediaPlayer.GetVisualizationData http://msdn.microsof...zationdata.aspx

Share this post


Link to post
Share on other sites

Why not just read the numerical values out of the .WAV file? (Assumption #2: You are using wave files.)


Even if assumption #2 is wrong, there is no reason why you could not manually open a .WAV file and parse it yourself to get the values out of the file, even if the actual sound you are playing is the same sound but in .MP3 format.


L. Spiro


I apologize, reading over this I don't think posting this while intoxicated was a good idea. Assumption 2 is sort of correct. I will be taking MP3 files and in a similar manner to audiosurf. Reading the MP3 (or other audio codec) and translating it into numerical outputs (laztrezort said it) using FFT or something similar. The problem is, I am not sure how to code something that will change an assortment of audio codecs into numerical outputs. Even if I have to use a seperate code for each and detect what the file type is, direct which code runs I need to be able to translate audio into numbers. Frequency, BPM, Pitch, Essentially take all the elements of an equalizer and measure them throughout the song taking samples every X milliseconds. For this project, since its on a mobile OS I am experimenting with unity. Unfortunatly, I have no idea what I am doing when it comes to the whole audio bit.

EDIT: Goal: User must be able to upload his or her own music and the code needs to accept that in one way or another in several common music formats and translate it into numbers one way or another. Edited by Hamaya

Share this post


Link to post
Share on other sites
I was going to suggest using ogg to avoid any licensing complications from mp3, but if you want your users to load their own mp3s you'll need a solution. Easiest may be to see if android / unity has an inbuilt library for converting mp3 to raw samples, that is fully licensed.

Essentially you want to use a third party library to convert from mp3 / ogg to raw data, probably 16 bit stereo or stereo floats. Then as said you could use something like FFT (either a library or find some source code), or you could simply do something like add up every 'window' of 1000 samples or find the peak over a window and use this loudness to determine your stuff happening in the game.

The more info you want to get out of the audio data, the more CPU it's going to use, which may be an issue targeting mobile phones. Things like extracting BPM and pitch are not trivial, but I don't think you really need to do that.

There have been games in the past that do the exact thing, I have a hazy memory of a spaceship shooter where the tunnel you are flying through is determined by the music playing, so it's very doable.

Share this post


Link to post
Share on other sites

The more info you want to get out of the audio data, the more CPU it's going to use, which may be an issue targeting mobile phones. Things like extracting BPM and pitch are not trivial, but I don't think you really need to do that.

There have been games in the past that do the exact thing, I have a hazy memory of a spaceship shooter where the tunnel you are flying through is determined by the music playing, so it's very doable.


The reason BPM was in there was that it would directly change the speed of the mobs spawned, making it any easy way to gauge the abstract difficulty of the song. For me at least, it is a rather vital gameplay component. And, I would be generating the entirety of the level before sending the user into it, which in theory should just give me a half a minute of load time, if optimized correctly. And then the spawns should be rather trivial as they are all predetermined at that point. What do you sugggest looking into to find a source or library?

Share this post


Link to post
Share on other sites
If that's to me I can only suggest google I'm afraid.

Never done it myself. Closest I've done is autocorrelation for finding pitch, which could conceivably work in some circumstances for bpm, but I wouldn't like to rely on it. I think it could be difficult for some types of music.

http://stackoverflow.com/questions/657073/how-to-detect-bpm-of-the-song-by-programming

Some bright spark on there points out that mp3 is already stored in the frequency domain. So if you are clever you could use the mp3 data directly rather than doing your own FFT.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
  • Advertisement
  • Popular Tags

  • Similar Content

    • By ethancodes
      I'm working on a system for my game that will allow the player to stack pick ups in a queue. As one pick up expires, the next automatically activates. I'm having an issue though where if I pick up the first one, it activates fine, but if i pick up a second directly after it, it overrides the first one, activates the second one, and then once it has run it's course, everything goes back to normal gameplay, no first pick up. I'm not sure why this is happening. Hopefully someone can spot what I'm doing wrong in my code.
      Here is the code for the pick up manager:
      // Update is called once per frame void Update () { if (pickUpQueue.Count != 0 && !pickUpActive) { pickUpActive = true; pickUpQueue[0].ActivatePickUp(); } DeactivatePickUp(); } void DeactivatePickUp () { if (pickUpQueue.Count != 0 && pickUpActive) { Destroy (pickUpQueue [0]); pickUpQueue.RemoveAt (0); pickUpActive = false; } } And here is the PickUp:
      public override void ActivatePickUp () { ball.GetComponent<Ball>().Speed = 2.0f; //increase ball speed... ball.GetComponent<Ball>().StartCoroutine(timer); //...set time that power up is active }  
      There is also a Base Pick Up:
      public void OnCollisionEnter2D (Collision2D collision) { Vector2 tweak = new Vector2 (Random.Range(0f, 0.2f),Random.Range(0f, 0.2f)); this.gameObject.GetComponent<Rigidbody2D>().velocity += tweak; //if the pickup makes contact with the paddle or ball.... if (collision.gameObject.tag == "Paddle" || collision.gameObject.tag == "Ball") { GameObject.FindObjectOfType<GameManager>().GetComponent<PickUpManager>().pickUpQueue.Add(this); Destroy(gameObject); //...and finally destroy power up object } } As a side note, I am trying to find a solution to this that will work for all of my pickups. Some pickups are ammo based, some are timed. 
    • By D34DPOOL
      Edit Your Profile D34DPOOL 0 Threads 0 Updates 0 Messages Network Mod DB GameFront Sign Out Add jobEdit jobDeleteC# Programmer for a Unity FPS at Anywhere   Programmers located Anywhere.
      Posted by D34DPOOL on May 20th, 2018
      Hello, my name is Mason, and I've been working on a Quake style arena shooter about destroying boxes on and off for about a year now. I have a proof of concept with all of the basic features, but as an artist with little programming skill I've reached the end of my abilities as a programmer haha. I need someone to help fix bugs, optomize code, and to implent new features into the game. As a programmer you will have creative freedom to suggest new features and modes to add into the game if you choose to, I'm usually very open to suggestions :).
      What is required:
      Skill using C#
      Experience with Unity
      Experience using UNET (since it is a multiplayer game), or the effort and ability to learn it
      Compensation:
      Since the game currently has no funding, we can split whatever revenue the game makes in the future. However if you would perfer I can create 2D and/or 3D assets for whatever you need in return for your time and work.
      It's a very open and chill enviornment, where you'll have relative creative freedom. I hope you are interested in joining the team, and have a good day!
       
      To apply email me at mangemason@yahoo.com
    • By davejones
      Is there a way to automatically change the start position of an animation? I have a bunch of animations set up on 3D models in unity. The issue is that I need to move the 3D models, however when I do so the animation start positions are not updated and I have to do it manually.

      Changing the transform of key frames is time consuming with the amount of animations I have, so I was wondering if there was a way to do it automatically?
    • By MoreLion
      hey all! We are looking for members for our Unity horror game! 
      Here’s the story:
      After a deadly virus plunges the world into chaos killing 85% of the human population there are now what they call “zones” these zones are watched very closely by the surviving government, people are checked every day for the virus, even if you touch the spit or any human waste or fluids of the victim who is infected, you will die. But one day, people in the west zone start to go missing, 1 woman goes outside the walls to uncover the mystery, is there more to the virus than meets the eye?, That is where your story starts.
      This game is not a long development game, I have loads other game ideas,
      I will also allow you to have a bit of creative freedom if you wish to add or share a idea!
      And no, it’s not a zombie game lol I feel like zombie games are too generic, in this game you will encounter terrifying beasts!
      There is some concept art one of our concept artists have made
      If interested email liondude12@gmail.com
    • By Canadian Map Makers
      GOVERNOR is a modernized version of the highly popular series of “Caesar” games. Our small team has already developed maps, written specifications, acquired music and performed the historical research needed to create a good base for the programming part of the project.

      Our ultimate goal is to create a world class multi-level strategic city building game, but to start with we would like to create some of the simpler modules to demonstrate proof of concept and graphical elegance.

       

      We would like programmers and graphical artists to come onboard to (initially) create:

      A module where Province wide infrastructure can be built on an interactive 3D map of one of the ancient Roman Provinces.
      A module where city infrastructure can be built on a real 3D interactive landscape.
      For both parts, geographically and historically accurate base maps will be prepared by our team cartographer. Graphics development will be using Blender. The game engine will be Unity.

       

      More information, and examples of the work carried out so far can be found at http://playgovernor.com/ (most of the interesting content is under the Encyclopedia tab).

       

      This project represents a good opportunity for upcoming programmers and 3D modeling artists to develop something for their portfolios in a relatively short time span, working closely with one of Canada’s leading cartographers. There is also the possibility of being involved in this project to the point of a finished game and commercial success! Above all, this is a fun project to work on.

       

      Best regards,

      Steve Chapman (Canadian Map Makers)

       
    • By Scouting Ninja
      So I have hundreds of moving objects that need to check there speed. One of the reasons they need to check there speed is so they don't accelerate into oblivion, as more and more force is added to each object.
      At first I was just using the Unity vector3.magnitude. However this is actually very slow; when used hundreds of times.
      Next I tried the dot-product check:  vector3.dot(this.transform.foward, ShipBody.velocity) The performance boost was fantastic. However this only measures speed in the forward direction. Resulting in bouncing objects accelerating way past the allowed limit.
       
      I am hoping someone else knows a good way for me to check the speed with accuracy, that is fast on the CPU. Or just any magnitude calculations that I can test when I get home later.
       
      What if I used  vector3.dot(ShipBody.velocity.normalized, ShipBody.velocity)?
      How slow is it to normalize a vector, compared to asking it's magnitude?
    • By Ds ds
      Hi, my name is Andres, I'm a programmer with a technician degree and a Diploma in C#, looking for a project in Unity to start my career in game development. I don't do it for a paid but a recognition and start a portfolio, preferably a 2D game. Thanks for read, have a nice day. 
       
    • By Victor Rodriguez
      Hi there! Is the first time that I'm posting here so I'm sorry if I'm doing it wrong ha. 
      So here it comes, my doubt is, I'm doing a game with different levels, each of these levels in one different scene. Each scene contains to cameras that you can change pressing a button. Everything works fine. 
      The only problem is that I would like it to look a bit more professional, and I would like that if you finish the level with camera2, the next level start the same way. I've been thinking about using dontdestroyonloadon both cameras, but obviously this cameras need to be attached to the player to make the movement work, what do you recommend? Sorry If I've explained it in a messy way, and feel free to dm me for anything. Thanks in advance! 
    • By Ike aka Dk
      Hello everyone 
      I am a programmer from Baku.
      I need a 3D Modeller for my shooter project in unity.I have 2 years Unity exp.
      Project will paid when we finish the work 
      If you interested write me on email:
      mr.danilo911@gmail.com
    • By markoal
      Hi,
      I'm Unity developer from Croatia and I'm looking to work on the paid project in my spare time.
      I have 5+ years of experience in Unity and I'm familiar with almost anything, including all platforms (also Switch, PS4 and Xbox).
      Feel free to contact me.
  • Advertisement
  • Popular Now

  • Forum Statistics

    • Total Topics
      631362
    • Total Posts
      2999570
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!