Alternative Visuals - Feedback Needed
Author
One factor that can be said to similar throughout almost all games, is that their visual medium is our visual medium; light. If one takes a look at a large cross-section of the recently produced games, one will see that they are graphically-centered around the visual medium in an attempt to replicate the world that we live in. While there are games that contravene this - focusing on gameplay or surrealism over photorealism - they have generally done this in the same fashion, that is, changing the priority and thus time spent. But there still exists a smaller niche of games which focus upon a visual medium completely alien to what we perceive normally; games such as Darwinia which creates a hypothetical 'digital world.'
With such a niche as a focus, the scope of Alternative Visuals is vast, but here I want to concentrate upon a single 'alternative' that is my goal. Instead of using a hypothetical visualization of a digital world, imagine replacing the visual medium of light, with sound. Sonar is something that has been a feature of a handful of games before, but to my knowledge it has never been the focus, which makes a physics-based sound navigation and ranging simulator (PoB-SNoReS?) game the first of its kind, applying ideals normally used in naval situations to normal settings.
The question here, is the ways that one could go about it. This game is still in the theoretical stage, as it should be. I, and the team with which I work, are engaged on a multi-stage project of which this game is a much later step, but the theoretics of this stage do warrant discussion.
Thoughts, comments, suggestions, and questions are welcomed and wanted; a focus upon the possible programming aspects, while not my forte, would still be appreciated.
I am an absolutely huge proponent of abstract or unique aesthetic styles for games. My work with unique styles thus far hasn't really branched out too far from the norm but, once I'm done with my current game, I want to start researching how to integrate aesthetic styles more akin to music visualizers than a typical 3D representation of a game or, if that doesn't work out, how to integrate audio/gameplay visualizations into a more traditional graphical approach.
That said, the kind of aesthetic style for a game depends largely on the kind of game that you want to make. There have been games in the past which have been based entirely on audio and, if I'm remembering this correctly, I believe one of these types of games made it into the Independent Games Festival or Student Showcase in the past few years. It may be worth looking through the archives.
The actual graphical representation of the game's audio, though, is a unique question that you have such a very wide variety of potential answers to. You could just have a set of visualizations for the audio that are completely independent of gameplay and purely serve as visual eye candy for a player to enjoy while playing the game; however, that's not an approach I'm fond of. You could have every sound in the game world manifest itself on the screen as a sort of in-game HUD/minimap hybrid.
Another viable alternative, and one I'm a fan of, is to have the actual game world be represented completely by the audio of objects. The louder an object is the larger and more dynamic it is represented as; static, immobile, and lifeless objects could be represented separately from everything else in the game world so as to alert a player to their presence and the type of object it is.
Really, though, you have so many options open to you for the visual representation of your idea that it's actually quite astounding.
That said, the kind of aesthetic style for a game depends largely on the kind of game that you want to make. There have been games in the past which have been based entirely on audio and, if I'm remembering this correctly, I believe one of these types of games made it into the Independent Games Festival or Student Showcase in the past few years. It may be worth looking through the archives.
The actual graphical representation of the game's audio, though, is a unique question that you have such a very wide variety of potential answers to. You could just have a set of visualizations for the audio that are completely independent of gameplay and purely serve as visual eye candy for a player to enjoy while playing the game; however, that's not an approach I'm fond of. You could have every sound in the game world manifest itself on the screen as a sort of in-game HUD/minimap hybrid.
Another viable alternative, and one I'm a fan of, is to have the actual game world be represented completely by the audio of objects. The louder an object is the larger and more dynamic it is represented as; static, immobile, and lifeless objects could be represented separately from everything else in the game world so as to alert a player to their presence and the type of object it is.
Really, though, you have so many options open to you for the visual representation of your idea that it's actually quite astounding.
Author
The current design scheme is to have a 3d world in which light is replaced with with sound. The key aspect that enters my mind, and is something you touched upon, is making it more than plain glitz. If memory serves, Alien versus Predator (the game) possessed a 'sonar mode' of its own, but it is something that I have not seen myself.
The real point of complexity though, is not the interaction of sound wave and surface, but sound wave and sound wave. Studies have shown that waves within water mimic the frequency and structure of sound waves to such a point that one can be used to study another. In such a manner, one can get a glimpse of the possibility of wave-to-wave interaction. Things such as wavefront propagation, wave canceling, and the pitch-reflection connection, come to mind as examples. As these aspects continue to pile, the task becomes greater and greater, requiring one to start first at one of the simplest aspects: Active Sonar.
Active Sonar is simple, in that it is a directed burst of sound, whose return rate is calculated and compared such as to register objects or surfaces. It acts as a 'beam' of sorts, used to search for specific objects within a narrow field. This, in effect, acts as the parallel to 'conic vision' or the central field of detail perceived by the eye. But beyond this, things become increasingly complex as you take into account the field of vision and so forth.
But as you can see, it is an intriguing concept. Combining both technical simulation and abstract, artistic gameplay. But it does pose another question. If you can see sound, can you still hear it?
The real point of complexity though, is not the interaction of sound wave and surface, but sound wave and sound wave. Studies have shown that waves within water mimic the frequency and structure of sound waves to such a point that one can be used to study another. In such a manner, one can get a glimpse of the possibility of wave-to-wave interaction. Things such as wavefront propagation, wave canceling, and the pitch-reflection connection, come to mind as examples. As these aspects continue to pile, the task becomes greater and greater, requiring one to start first at one of the simplest aspects: Active Sonar.
Active Sonar is simple, in that it is a directed burst of sound, whose return rate is calculated and compared such as to register objects or surfaces. It acts as a 'beam' of sorts, used to search for specific objects within a narrow field. This, in effect, acts as the parallel to 'conic vision' or the central field of detail perceived by the eye. But beyond this, things become increasingly complex as you take into account the field of vision and so forth.
But as you can see, it is an intriguing concept. Combining both technical simulation and abstract, artistic gameplay. But it does pose another question. If you can see sound, can you still hear it?
I had a similar idea a while ago where I was considering how I could allow a certain game to be accessible to blind players. Nothing ever came of it, but I was reminded of it when I saw Be The Wumpus on happypenguin.org, and of course again with this thread.
Anyways, aside from providing an accurate simulation of how sound works it seems to me that you would also have to take care to train the player properly early on so that they won't be overwhelmed by the complex interactions of sound and environment later on.
Anyways, aside from providing an accurate simulation of how sound works it seems to me that you would also have to take care to train the player properly early on so that they won't be overwhelmed by the complex interactions of sound and environment later on.
Author
Or is it that if you see sound, you hear color?
The level of visual complexity is a hard aspect to manage; how much is too much? To use an example, determining the field of vision outside of the conic vision is pretty much trial and error; you don't hear in one direction, so how does that effect what you see? What this boils down to is a fish eye effect, distorting sounds which take place beside the player, but how much is too much? At what point is the ping rate of active sonar too fast? At what point is wavefront interaction too confusing?
As the questions build, one factor stands out; how do you imagine the visualization of sound? The aim is very much a niche market, which means that that market needs to have input on the connecting game. Thus, speak your mind unto me.
The level of visual complexity is a hard aspect to manage; how much is too much? To use an example, determining the field of vision outside of the conic vision is pretty much trial and error; you don't hear in one direction, so how does that effect what you see? What this boils down to is a fish eye effect, distorting sounds which take place beside the player, but how much is too much? At what point is the ping rate of active sonar too fast? At what point is wavefront interaction too confusing?
As the questions build, one factor stands out; how do you imagine the visualization of sound? The aim is very much a niche market, which means that that market needs to have input on the connecting game. Thus, speak your mind unto me.
The first thing that came to my mind was Daredevil, the comic-book superhero as well as its movie starring Ben Affleck & Jennifer Garner (as Elektra). Daredevil's a blind guy who relies on sound as his primary form of sensory input, almost to a superhuman level.
If some developer decided to make a Daredevil game, I could very well see such a sonar mechanic used, illuminating hallways, corridors, and especially badguys.
How I would do this if I were to do this is to keep in mind that sound travels slower than light, therefore the info you get from your surroundings will have some lag or delay compared to optical input, with different speeds depending on the medium (through vacuum - none, through air - slow, through solids - pretty fast). You could have two types of info input, Sonar Surface-Echoing to which illuminates your surroundings by displaying sound waves (circles that grow bigger over time) bouncing off of surfaces and objects, and then the actual Sonar Emissions which reveal the positions of sound makers (where the circles came from) such as people, animals, gunshots, cars, etc. You could also also offer the players the ability to set variable levels of sensitivity/focus so that they could get a wide level of information, from the very soft & specific (such as people's heartbeats to predict their actions) to the very loud & general (their footsteps and voices).
It would be pretty interesting to see this implemented.
[Edited by - Tangireon on September 17, 2008 7:24:40 AM]
If some developer decided to make a Daredevil game, I could very well see such a sonar mechanic used, illuminating hallways, corridors, and especially badguys.
How I would do this if I were to do this is to keep in mind that sound travels slower than light, therefore the info you get from your surroundings will have some lag or delay compared to optical input, with different speeds depending on the medium (through vacuum - none, through air - slow, through solids - pretty fast). You could have two types of info input, Sonar Surface-Echoing to which illuminates your surroundings by displaying sound waves (circles that grow bigger over time) bouncing off of surfaces and objects, and then the actual Sonar Emissions which reveal the positions of sound makers (where the circles came from) such as people, animals, gunshots, cars, etc. You could also also offer the players the ability to set variable levels of sensitivity/focus so that they could get a wide level of information, from the very soft & specific (such as people's heartbeats to predict their actions) to the very loud & general (their footsteps and voices).
It would be pretty interesting to see this implemented.
[Edited by - Tangireon on September 17, 2008 7:24:40 AM]
Quote:Original post by Elhrrah
Or is it that if you see sound, you hear color?
I'm not sure color is required to be part of your sensory inputs if you are able to "see" sound waves; what is most important I think is that you are displaying spatial shapes & positions using sound waves, which can be of the bouncing-off kind (ex: illuminating hallways) as well as the emitting kind (ex: someone is talking). Color then, or rather what I would suggest instead as brightness, could denote sound strength/age, how close the waves are to their source & how far it has traveled (the younger or more recently produced the sound wave is, the brighter it will be depicted).
Quote:The level of visual complexity is a hard aspect to manage; how much is too much? To use an example, determining the field of vision outside of the conic vision is pretty much trial and error; you don't hear in one direction, so how does that effect what you see?
You could do what many first-person spaceship shooter games use, and that is to display targets that are behind you as icons on the sides of the screen, in the direction where it came from, so that when you go and trace that icon with your centered crosshairs, you will be turning towards that target and eventually facing it.
Quote:What this boils down to is a fish eye effect, distorting sounds which take place beside the player, but how much is too much?
For displaying targets that are behind you as icons on the sides of your screen, show them as icons/pictures rather than their sound waves so you can immediately distinguish what is behind you from what is simply on the sides of your vision in front of you. These icons could be in the form of pictures, which display what that sound is, such as a cat, footsteps, a passing car, people's dialogue, etc. The solidity/transparency or brightness of that icon then can tell you how loud or close to you that source is (the more solid the icon, the louder or closer it is).
Quote:At what point is the ping rate of active sonar too fast? At what point is wavefront interaction too confusing?
From what I know of sound physics, is that the ping rate is very much the same as the frequency of the sound wave, or its pitch. Deeper notes will have more spaced-out waves, while higher notes will have tighter waves representing higher frequencies. If you have many sound sources in your world (not just your radar), then they all might vary in their frequencies.
Quote:As the questions build, one factor stands out; how do you imagine the visualization of sound? The aim is very much a niche market, which means that that market needs to have input on the connecting game. Thus, speak your mind unto me.
Well first of all, there are the sound sources, which you could represent on your screen as a series of spheres (or to you, as circles) emitted from that source that grow larger over time (and eventually fade away) to which traverse over a length of space. The space between each surface of each sphere represent that sound's frequency or tonal pitch (tighter spaces = higher frequency/pitch, while lengthier spaces = lower frequency/pitch). Spheres/circles that just have been emitted from their source would be brighter than circles that have traveled far from its source. Additionally you could also represent the Doppler effect for moving sound sources by using this spheres/circles model.
Then these sound waves bounce off of surfaces or objects so you can distinguish what your environment looks like. For the purposes of clarity, at the moment when the circles/waves bounce off of a surface, make them brighter so as to distinguish that type of information from other circles/waves so that the player can more easily see what their environment looks like.
As you could imagine, your screen could be filled with several hundred spheres/circles at once, making it pretty confusing to the player. So then what it all boils down to in terms of clarity of the information displayed on screen is the brightness of the circles/waves depicted - not only can brightness prevent info jumbling and confusion, you can immediately tell what waves are important to you ("young waves" - those that have just bounced off of something or have just been emitted from a source) from those that aren't as important ("old waves" - waves that have traveled for some time from their source and haven't interacted with anything yet).
[Edited by - Tangireon on September 18, 2008 5:35:14 AM]
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
