Hello,
I'm currently studying machine learning and neural networks. While studying theory is indispensable, I want to immediately apply it to a fun project. I have intermediate programming skills (just no experience with NNs), so I want to write a simple game and create a neural network-controlled bot to play it. While I understand that a NN may not be the best approach to creating a game playing actor, I am still willing to do it.
I am trying to come up with a game simple enough to be treated as a set of numeric inputs for the NN. I also want to write it from scratch to have complete control. Right now I think that I should start with a Pong game, since this game can be interpreted simply as positions of paddles and a position/direction of a ball. Then I should probably move on to something like Space Invaders or Pac-Man or Battle City (or maybe even Black Jack?).
What would you suggest?
A game suitable to training a neural network?
- Converting the ML AI's intentions into something the game can respond to is hard
- Getting data into the ML layer so the AI knows what to reason about is even harder
- Training the ML layer with appropriate feedback based on the first two elements can get extremely hard
There are members here who have written checkers AI with NN backing, for example; and like you say, most of the time ML isn't the best solution. But it is certainly an educational and entertaining project.
I'd suggest focusing on games that make the first two problems as easy as possible, so you can concentrate your efforts on the third problem.
A 2D grid you move a mouse on and the neural network controlled 'follower' tries to follow the mouse pointer
its graphical so visually shows the activity (and can be 'played'/demonstrated)
its easy to generate training sets for
can be as simple as JUST directional guidance/control based on the XY cooordinate of the object and the current pointer
can be made more complex a task if 'blocking' objects (cells) can be on the grid (even moreso if they are walls instead of just dodging obstructions (note that grid edges can come into play as 'environment' in these cases)
Increase the size of the grid til its beyond practical 'size of the NN layer' to demonstrate predigesting relative inputs.
I'd suggest you start with these:
Classic board games like checkers/othello. (I am not going to recommend chess or go due to its high branching factor)
Tic-tac-toe
The hard part of using NN in games is training it because games are interactive and have animations. Once your game start using some kind of animation in which the NN has to respond at certain frame doing certain things, that's when it gets tricky. Your game must be coded in such a way that it can be run in a endless loop with just the NN playing it.
The games above can be simulated easily without graphics, so that's why I recommend to start with those.
I'd suggest focusing on games that make the first two problems as easy as possible, so you can concentrate your efforts on the third problem.
I will definitely keep that in mind.
The hard part of using NN in games is training it because games are interactive and have animations. Once your game start using some kind of animation in which the NN has to respond at certain frame doing certain things, that's when it gets tricky. Your game must be coded in such a way that it can be run in a endless loop with just the NN playing it.
That's exactly what I am going to do. I want to have a game "engine" that will compute the state of the game world for each step and send it to the NN, then send computed state with NN's output to a renderer that will display the state of the world plus NN's actions.
A 2D grid you move a mouse on and the neural network controlled 'follower' tries to follow the mouse pointer
That looks even simpler than a Pong game. Good suggestion, I'm going to start with this one.
Thanks for the great answers!
A racing game with simple graphics and rules would be slightly more complex, but much more fun for humans, than the suggested cursor chasing.
Simple 3D or pseudo-3D graphics from the point of view of the vehicle might allow the use of screenshots as input, like in most experiments with Atari 2600 games, but a more abstract input might be more effective for the NN and more convenient for designing graphics.
For example, if the course is represented as a "tube" of passable terrain around a center line, you can adopt a coordinate system consisting of arc length (between the projections on the center line of the current player position and of the point of interest) and distance from the center line; the NN input can then consist, in addition to car positions and velocities, of track shape given as curvature of the center line and width at many predetermined arc length offsets ahead of the player.
