Neural Nets
Author
Neural Nets... The term sounds like something mysterious and highly interesting. I have studied them a bit just out of curiousity, but I am not sure if I can think of any was to apply them in games. I know there are some, but I am just too idiot to figure them out.
So, have any of you used Neural Nets in your game(s)? For what? What were the results? Are you sane? Are Neural Nets of any use for a game programmer? Reply anything (as long as it involves the subject)!
[EdNote: rk talks too much.]
rk
Nah, although I have (often) ranted about neural networks in the past, I have never found a need to use then in a game.
The main problem I have with implimenting them is that they are sloooowww and with game restrictions there's just no point.
However, I think that they certainly worth looking at, as processor time gets faster, and more emphasis is put on creating better AI, I think that neural networks will become more popular for games, simply cause they inherently have the ability to develop, which I think could extend the longetivity of games enormously!
Imagine a game like DeusEx where you play with different styles, but as you get through the levels the bad guys know how you like to play, and start, for example, keeping an eye out for snipers, and staying in cover, if the player's playing a very long distance game.
At the moment though, there's not much point to use much beyond scripting!
Edited by - Nutter2000 on March 19, 2001 10:50:34 AM
The main problem I have with implimenting them is that they are sloooowww and with game restrictions there's just no point.
However, I think that they certainly worth looking at, as processor time gets faster, and more emphasis is put on creating better AI, I think that neural networks will become more popular for games, simply cause they inherently have the ability to develop, which I think could extend the longetivity of games enormously!
Imagine a game like DeusEx where you play with different styles, but as you get through the levels the bad guys know how you like to play, and start, for example, keeping an eye out for snipers, and staying in cover, if the player's playing a very long distance game.
At the moment though, there's not much point to use much beyond scripting!
Edited by - Nutter2000 on March 19, 2001 10:50:34 AM
Neural nets are a good idea, but if you want a character to learn a better bet would be to use a reinforcement learning technique. That is you give the NPC a goal (Eg. to shoot another NPC) and everytime they do that they receive a reward and are more likely to do that again in the future.
Implementing it would be far quicker than using a neural network as updating a neural network takes forever. With this method you have a current state and an action and everytime you perform an action that reaches the goal, you make that action more likely in that state. It is simply a matter of storing a set of states and how good each action is in each state.
---
Fleejay
Implementing it would be far quicker than using a neural network as updating a neural network takes forever. With this method you have a current state and an action and everytime you perform an action that reaches the goal, you make that action more likely in that state. It is simply a matter of storing a set of states and how good each action is in each state.
---
Fleejay
They could be used without much hassle in turn-based games. You could even have it sit and update them during your turn, it''s not like that''s a very intensive period of the game...
"This album was written, recorded and edited at Gröndal, Stockholm in the year of 2000. At this point in time money still ruled the world. Capitalistic thoughts were wide spread. From the sky filled with the fumes of a billionarie''s cigar to the deepest abyss drenched in nuclear waste. A rich kid was a happy kid, oh..dirty, filthy times. Let this be a reminder."
- Fireside, taken from back of the Elite album
"This album was written, recorded and edited at Gröndal, Stockholm in the year of 2000. At this point in time money still ruled the world. Capitalistic thoughts were wide spread. From the sky filled with the fumes of a billionarie''s cigar to the deepest abyss drenched in nuclear waste. A rich kid was a happy kid, oh..dirty, filthy times. Let this be a reminder."
- Fireside, taken from back of the Elite album
You can use a simple sort of BPN (Back Propagation Network) to apply feedback in certain situations to make the game elements seem to react with more intelligence. For example, if you set up a BPN with some fuzzy logic built into the output you can make a monster take certain actions based on the current state. If you''ve seen the code for a BPN and compared that to a FSM (Finite State Machine), which seems to be the standard, you will see that the FSM is actually more complicated. This is due to the fact that a seemingly minor change in the state handling code for one state can have a huge impact on the rest of the FSM. Using NN''s (Neural Nets) one can map differing states to inputs of other states without interfering with the states that have nothing to do with the one being changed. Whew! That was obfuscated for sure! Then there is the BAM (Binary Associative Memory) network which can actually remember different patterns that your monsters learn. You can then use the BAM with some genetic algorithms to make a monster that can take on Big Blue!
This is a reply to Fleejay''s post about reinforced learning. I just wanted to point out that what you described can be implemented using an Adeline network with a ''Mexican Hat'' threshold function. This will reinforce certain behaviours and weed out others. If you couple this with some simple genetic algorithms and a BAM to store your results you can have a real powerhouse of learning.
The problem with using an adaline network (or any neural net for that matter) is the recalculation of the weights, for a large network (which you would need to do any sort of problem solving) you need to back propagate the error and recalculate the weights for each update. This isn''t viable in a realtime game. The reinforcement method however will look at the actions taken and how well the string of actions performed in the game environment, it is then simply a matter of updating some state-action probabilities, without the need to do error back propagation. Basically what I''m saying is reinforcement learning is more suited to real time games than neural nets are.
---
Fleejay
---
Fleejay
I tried to have a backprop NN learn from my game play. Basically, I had as inputs my own stats (3D position, ennemy''s position relative to me, etc) and as output my own reactions (move right, shoot, etc). This is what the NN was trained on.
Then the NN would do the same for each AI-controlled character, without learning, just acting as I would if I were them. Or so I hoped to see... Naive I was at that time, but I was still curious to see how much it would learn...
It did learn something, but what it learned was not useful. The AI-controlled characters would move in one direction and jump uncontrollably. The NN completely missed the subtleties of the gameplay, even if I trained it several times with different parameters and much playing. It simply didn''t have the capacity to learn high-level strategies. Jumping and moving is good for a beginner to survive a bit longer, but you need a lot more to be an expert player.
I still think NNs can be used, but with proper pre- and post-wrapping. Basically, you need to do the work of reducing the data to a minimum and make it digestable to the NN. Same applies to its responses.
This is a project I keep for the near future...
Then the NN would do the same for each AI-controlled character, without learning, just acting as I would if I were them. Or so I hoped to see... Naive I was at that time, but I was still curious to see how much it would learn...
It did learn something, but what it learned was not useful. The AI-controlled characters would move in one direction and jump uncontrollably. The NN completely missed the subtleties of the gameplay, even if I trained it several times with different parameters and much playing. It simply didn''t have the capacity to learn high-level strategies. Jumping and moving is good for a beginner to survive a bit longer, but you need a lot more to be an expert player.
I still think NNs can be used, but with proper pre- and post-wrapping. Basically, you need to do the work of reducing the data to a minimum and make it digestable to the NN. Same applies to its responses.
This is a project I keep for the near future...
Supervised learning methods in neural networks are very linear and limited. For a start you have to know the error in the system at a given point in time, then backpropagate it through the network. Working out the amount of error for an output for a game of any complexity is an artform. Imagine making a move in an RType style game, seeing how the enemy reacts and transducing that into a floating point number to represent the ''error'' for one output. I can''t even see how that can be done. Besides this, the intent of backpropagation is to minimize the error over the network. By using gradient descent (differential of the Error with respect to a weight) all backprop does is get the minimum average error over the network. None of the neurons or weights know the optimum position to be in so they fall into the nearest local optima which is bound to be non-optimal. They can also only be used on feed forward layered, non recurrent, non-dynamic networks. Which means they are limited in their spatial and temporal dynamics, two very important factors in a realtime game.
Personally I would use genetic algorithms to train the nets before release and have no (or minimal) training (via GA''s) within game time.
Just my humble opinion.
Mike
Personally I would use genetic algorithms to train the nets before release and have no (or minimal) training (via GA''s) within game time.
Just my humble opinion.
Mike
Tell me more about the GA method. I understand then you evolve the nets originally from a bunch of random genes, selecting the best using a fitness criteria, but how do you go about changing the genes while the game is running?
Using a reinforcement technique is a similar idea, but instead of genes you have state-action probabilities. Eg you could have a state "someone is shooting at me from infront" and an action could be "turn to face the target and shoot back". If this is successful (ie if you kill the target) the probability that you perform that action in that state becomes greater.
Most of the time you perform the action with the greatest probability in that state, but occasionally (1-10% of the time) you perform an action that isn''t as good so that you can learn. It may not have been such a good action in the past, but it might be now.
This technique is fast. You have to remember all of the actions you perform from some time in the past, but then when you reach a goal (eg you shoot someone or you get shot - or even more simplier like you dont walk into a wall) you update the probability of all the state-action pairs from the arbitrary time to the current time, and you reset your action list.
I don''t think this technique has been used succesfully in a realtime game (Black and White uses something similar?), although it has been used to build a backgammon player considered one of the best in the world.
---
Fleejay
Using a reinforcement technique is a similar idea, but instead of genes you have state-action probabilities. Eg you could have a state "someone is shooting at me from infront" and an action could be "turn to face the target and shoot back". If this is successful (ie if you kill the target) the probability that you perform that action in that state becomes greater.
Most of the time you perform the action with the greatest probability in that state, but occasionally (1-10% of the time) you perform an action that isn''t as good so that you can learn. It may not have been such a good action in the past, but it might be now.
This technique is fast. You have to remember all of the actions you perform from some time in the past, but then when you reach a goal (eg you shoot someone or you get shot - or even more simplier like you dont walk into a wall) you update the probability of all the state-action pairs from the arbitrary time to the current time, and you reset your action list.
I don''t think this technique has been used succesfully in a realtime game (Black and White uses something similar?), although it has been used to build a backgammon player considered one of the best in the world.
---
Fleejay
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
