How to make bot playing game with neural net

Started by
6 comments, last by IADaveMark 9 years, 1 month ago
i have a trained neural net for playing enemy-escaping game(bot mode), so the inputs are positions of player and of enemy and 4 outputs for actions values (left,right,up,down). Action with maximum value is choosen for move.
But what if i have not one but 3 or more enemies in game?
Here is my idea:
1. estimate neural net output(actions values) for each enemy, add outputs to 4-d array.
2. choose action with maximum value from array.
Is it correct?
Advertisement
I don't understand your idea. The neural network should have inputs that correspond to what the agent perceives. If there are three enemies and it can perceive all of them, use enough inputs to represent that.

I don't understand your idea. The neural network should have inputs that correspond to what the agent perceives. If there are three enemies and it can perceive all of them, use enough inputs to represent that.

What if number of enemies 100? I dont think that making such a big neural net is a good idea. May be iterate all enemies through the small neural net and after that deciding what action to take based on cumulative output?

If the number of enemies is large, you probably want to summarize it by dividing the space around the agent in a few regions (say, 30) and then use the number of enemies in each region as your input. That would work well for any number of enemies.

What is your terrain mechanism like and how much 'generalization' are you willing to accept ?

Does the Escaping decision also have to take into account terrain features which may invalidate certain directions of movement?

Are there 'friendly' units that would be good to move towards as a preference???

Is the movement system on a regular grid with 1 square progression of movement - 8 possible neighbor directions of movement? (something of that sort ????)

Even with more detailed terrain, the situation can be simplified into a graduated unitized form like that (there are discrete 'neurons' in most NN systems ....)

I assume the 'escape' is to aim for the most open area around the thinking object and when there are multiple enemies its attempting to be far away from any 'enemies' as the goal....

How big a grid ? (size of are is considered for the decision (its possible also to make it more like a system of concentric pie slices instead of being grid based if you have large distances (too huge to grid it) with some kind of power of 2 concentric circle progression which the enemy positions will be judged as (and a pie slice angle division that can represent more than 8 X 45 degree increments and movement directions).

You will need training data to form your NN - sets of positions and the "right" direction to go

and likely some normalization/fractioning of the each marker signal strength as your enemy count goes up (so the same net can be used with a varying number of enemies)

You can simplify the NN sizing by despinning/aligning the NN as a relative overlay (say with North being the exact position of the first enemy and the others being in relation to that first ( that eliminates/simplifies alot of duplicate positioning combinations around the circle centered on the thinking object and allows fewer or better coverage of the patterns.)

--------------------------------------------[size="1"]Ratings are Opinion, not Fact

The four outputs are probably wrong, as a large part of the output space (going both up and down or both left and right) is nonsensical. You could instead adopt two continuous-valued outputs, x and y velocity, which could have both signs to represent all directions of desired movement and could be jointly quantized in arbitrary ways to move along a grid or another type of pathfinding graph.

Omae Wa Mou Shindeiru

The four outputs are probably wrong, as a large part of the output space (going both up and down or both left and right) is nonsensical. You could instead adopt two continuous-valued outputs, x and y velocity, which could have both signs to represent all directions of desired movement and could be jointly quantized in arbitrary ways to move along a grid or another type of pathfinding graph.


I don't think there is anything wrong with having those four outputs, if those really correspond to the actions available. See for instance what DeepMind did with Atari 2600 games (paper) (video).

Or just use a freaking influence map and be done with it.

Dave Mark - President and Lead Designer of Intrinsic Algorithm LLC
Professional consultant on game AI, mathematical modeling, simulation modeling
Co-founder and 10 year advisor of the GDC AI Summit
Author of the book, Behavioral Mathematics for Game AI
Blogs I write:
IA News - What's happening at IA | IA on AI - AI news and notes | Post-Play'em - Observations on AI of games I play

"Reducing the world to mathematical equations!"

This topic is closed to new replies.

Advertisement