Kanoue_Yuuma

Neural Network AI Discussion Group

Recommended Posts

Hello everyone. I am a new member of the community that just signed up. While I was browsing the neural network AI topics, I found lots of questions on how to implement neural networks in game AI. I am very interested in this topic as well and was wondering if there is a community that exists for discussing approaches and applications to using neural networks in game AI. If so can someone kindly direct me to it? If not can we make one?

Share this post


Link to post
Share on other sites
Neural networks are a tool that has had limited application in game AI. I don't think it's worth having a separate venue to talk about them. Try posting here and see how it goes.

Share this post


Link to post
Share on other sites

Neural networks and reinforcement learning take a large pool of data sets.  They are also impossible for a game designer to fine-tune.  State machines (and nested state machines) are the typical route for games.

 

Some games have used reinforcement learning techniques, although generally not to the point of an ANN. Black and White used it to help train your beast avatar, you could reward or punish the creature and it would push it to repeat or avoid the actions in the future. Some board games like Go and Chess simulators have used ANN's, it works well against casual players but not the masters of the games.

 

Some features of games, like voice recognition or text recognition or video capture recognition or VR gesture recognition, these sometimes are trained on a large data set to recognize the elements, but those don't happen during the game and the training results generally don't directly affect the game. Instead the results are used as input -- often validated by confirmation -- and consumed by the simple state machines that run the game.

Share this post


Link to post
Share on other sites

Some board games like Go and Chess simulators have used ANN's, it works well against casual players but not the masters of the games.


This is not true for go. All of the strong go programs out there use ANNs, and the best one (AlphaGo) seems to be stronger than any human.

Share this post


Link to post
Share on other sites

 

Some board games like Go and Chess simulators have used ANN's, it works well against casual players but not the masters of the games.


This is not true for go. All of the strong go programs out there use ANNs, and the best one (AlphaGo) seems to be stronger than any human.

 

Indeed, Alphago uses a mix of classic AI techniques (Monte Carlo search tree) and ANNs. It could easily have been done with chess too, and it worked in quite a lot of Atari games.

No Limit Hold Em Poker heads up too has been beaten by an ANN based AI.

The problem is that many games seem much more complex than go, or don't require a killer AI.

The games that suffer the most from poor AI would be wargames, and complex strategy games like Civilization, Starcraft and XCOM, but the game state would be so complex that using ANNs for them might be quite difficult (but their "classical" AI is usually not very good either). 

It might work but it would be mostly uncharted territory.

Share this post


Link to post
Share on other sites

[...] It could easily have been done with chess too, [...]


Actually, I am not so sure. There have been several attempts at using neural networks for chess, and the results are generally poor. I've tried myself without much success. My last attempt was this past weekend, when I tried to train a very simple ANN to learn the value of material imbalances (including subtleties like the strength of the pair of bishops, the loss of value of the bishop with the presence of lots of your own pawns, etc.). I couldn't get it to work better than a few hand-crafted rules, and that seems to always be the story with chess.

Share this post


Link to post
Share on other sites

No Limit Hold Em Poker heads up too has been beaten by an ANN based AI.

Kind of.

The problem is that many games seem much more complex than go, or don't require a killer AI.
 

This.

People need to remember that we aren't trying to "solve" something here... we are trying to create an experience. Often, that experience needs to include intentionally sub-optimal decisions and intentionally distinct characters. If you are trying to "solve" behavior, you get neither.

Share this post


Link to post
Share on other sites

 

[...] It could easily have been done with chess too, [...]


Actually, I am not so sure. There have been several attempts at using neural networks for chess, and the results are generally poor. I've tried myself without much success. My last attempt was this past weekend, when I tried to train a very simple ANN to learn the value of material imbalances (including subtleties like the strength of the pair of bishops, the loss of value of the bishop with the presence of lots of your own pawns, etc.). I couldn't get it to work better than a few hand-crafted rules, and that seems to always be the story with chess.

 

I thought Giraffe had achieved reasonable success with a NN based approach.

Alphago performed much worse when it relied solely on ML. Combining regular search and ML worked best acording to the deepmind go team. So maybe combining both would also work better for chess. That said, it is also quite possible that raw computation outperforms ML for chess because it lends itself to brute force better.

Share this post


Link to post
Share on other sites

 

The problem is that many games seem much more complex than go, or don't require a killer AI.
 

This.

People need to remember that we aren't trying to "solve" something here... we are trying to create an experience. Often, that experience needs to include intentionally sub-optimal decisions and intentionally distinct characters. If you are trying to "solve" behavior, you get neither.

 

 

But actually, if we were to separate a civilization AI into several subproblems, like:

  • which goal to pursue to reach victory conditions
  • alliances and wars
  • economy, research and production
  • unit task/theater assignment
  • unit micromanagement

Using ANN for diplomacy would be horrible, and economy lends itself much better to a search or utility approach, but wouldn't it make sense to use deep reinforcement learning for the unit micromanagement part?

It is the one that resembles zero sum board games the most, and its space state is quite huge, but not orders of magnitude larges than go (if we decide to limit it to the immediate surroundings of a unit and not the whole world map). It is also something regular game AI usually has trouble to handle.

 

 

I thought Giraffe had achieved reasonable success with a NN based approach.


No, Giraffe's Elo on CCRL 40/4 is 2411, which is quite mediocre. There are many engines that are hundreds of Elo points stronger, including my own, RuyDos.

 

Nice! Is there any place where you have documented your work on it?

Share this post


Link to post
Share on other sites

It is the one that resembles zero sum board games the most, and its space state is quite huge, but not orders of magnitude larges than go (if we decide to limit it to the immediate surroundings of a unit and not the whole world map). It is also something regular game AI usually has trouble to handle.
 

When you consider multiple unit types and even multiple upgrades of units, it gets out of hand fairly quickly.

Share this post


Link to post
Share on other sites

That is a fair point. But trimming the features down to a reasonable level is what we already do with utility functions. So what would be the problem with using ANN to fine tune the contributions of the utility function?

Some behaviours could be very hard to grade indeed, but some other not so much (like preventing the player characters from reaching their objective, or inflicting as much damage as possible before dying for a NPC).

Share this post


Link to post
Share on other sites

If you have a fitness function that can tune an ANN to give you optimal play, why don't you just use the fitness function directly to evaluate candidate state changes?


I doubt he has that. Sometimes all you have is a database of previous games, where you have game situations you can feed into a neural network and have it try to predict the outcome of the game. You train an ANN to estimate the expected reward at the end of the game, which can then be used in a utility-maximization setup.

Share this post


Link to post
Share on other sites

Indeed, I was only planning to score the final state for each character when he dies, but that is a good point. Even grading the utility of things accomplished by a given character through his lifetime could be hard to get right compared to only grading the final outcome for the whole team.

But I think it would still be "easier" than finding a good metric to grade the utility at each time.

Edited by Galdred

Share this post


Link to post
Share on other sites

People need to remember that we aren't trying to "solve" something here... we are trying to create an experience. 

I'm far more excited for applications beyond solving games. Neural networks can generate images, videos or sounds. This could make every NPC's voice unique (looking at Skyrim, with it's 5 voice actors), make realistic animations (Spore creatures could evolve how they walk) or for example generate new biomes for Minecraft. Open world games could become much more diverse if neural networks can fill in the details. It's procedural generation that can actually work.

Are there any discussion groups for that?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now