How to use reinforcement learning to optimize the computer AI of turn-based strategy games

Started by
3 comments, last by alvaro 1 month, 1 week ago

I tried running a project inspired by AlphaZero's Gomoku AI, which can be found at https://github.com/junxiaosong/AlphaZero_Gomoku.

I am wondering if it is possible to use the idea of this project to build computer AI for some turn-based strategy games. These strategy games have a 2D grid map, and all characters will occupy one grid. Their turns alternate, and they can use different actions each turn.

The input design of Gomoku or AlphaGo is very interesting. It is divided into several different slices of input neural networks, including the positions of white and black pieces, who goes first, and the positions of the recent moves refer to the past. If it is a two-player strategic game, input the positions of both sides separately, and then input the first and second moves separately. That's roughly it. One additional question is how to represent different NPCs, as chess games typically have identical pieces. One possible solution is to use combat power to represent them, by predicting combat power through numerical values. This way, it can simplify the representation.

However, this definitely involves fitting, because compressing high dimensions into one dimension in order to optimize can only adjust the combat power based on the perspectives of different non-player characters.

Advertisement

hxyxyz said:
I am wondering if it is possible to use the idea of this project to build computer AI for some turn-based strategy games.

Moving thread to the Artificial Intelligence forum based on that quote and the post's title.

hxyxyz said:
One additional question is how to represent different NPCs, as chess games typically have identical pieces.

If you want feedback or have questions on your game's design, the Game Design & Theory forum is still the best place for that.

-- Tom Sloper -- sloperama.com

Is it possible? Sure, there are lots of AI approaches you can use.

hxyxyz said:
It is divided into several different slices of input neural networks, including the positions of white and black pieces, who goes first, and the positions of the recent moves refer to the past.

That's an unusual way of doing it, but I guess if you're trying to build an AI that way it might eventually get trained to do something useful. I doubt it, but the machines are very good at eventually finding something in the patterns for an action. Games rarely use reinforcement learning for AI, but a few have done it, and a few automatic players have also done it over the years.

Personally as there is access to source code I'd instead go with an enumeration of possible moves and actions, computing weights based on utility functions, and going from there, but if you've got time and inclination to go for reinforcement learning models instead, I suppose why not go for it.

Using RL is an interesting project and can result in very strong AI, but projects like AlphaZero take very large amount of computation.

I don't understand how your NPCs are different from chess pieces. Is it that there many types of them? Or perhaps that they have stats? Those seem like small obstacles.

If the players have incomplete information about the state of the game, that would change things.

Whatever you do, it helps a lot to separate the representation from the game logic, and a very good way to do that is by defining a text protocol that the engine and the GUI use to communicate. Then the engine can be easily tested, matches can be played automatically… Check out the UCI protocol for chess, or GTP for go. You may want to also define a format for saving games (like PGN for chess).

This topic is closed to new replies.

Advertisement