Blind Poker Artificial Neural Network

Started by
31 comments, last by sjhalayka 6 years, 8 months ago

@Kyltotan: Rather than test for error, I could abort after x number of games, or some high enough win/loss ratio. Indeed, I'm having trouble knowing when to stop the learning process. Millions of games would be played, for sure, seeing how the 5 card odds of landing a royal flush in Texas Hold 'Em is like 649,739:1.

@alvaro: I think I see what was meant by knowledge in advance: the training process must check to see if the proposed state change given by the ANN is even valid. Thus, all of the possible state changes must be enumerated, so as to have something to check the validity of the proposed state change. If the proposed state change is invalid, then one valid state change is picked pseudorandomly from the enumeration of possible state changes. Anyway, the main point is that the possible state changes must be known to the AI in advance. That's cheating, I suppose, but it's the only way that I can see it working. I can also see this being part of the equation when it comes to the socket server having to check to see if a player's proposed state change is valid.

Advertisement
15 hours ago, sjhalayka said:

Practical Neural Network Recipes in C++

This book is 24 years old. The state of the art has advanced SIGNIFICANTLY since then, not to mention the computing power available. Do not use that book as an example of what to do today.

In particular, genetic optimisation of neural networks made sense in the 90s; it mostly doesn't, now.

 

12 hours ago, sjhalayka said:

Rather than test for error, I could abort after x number of games

This sounds like you don't really understand what the inputs and outputs of the net should be. If you expect to train the network by running entire games and then scoring it with a Win or a Lose, you're throwing away tons of useful information and wasting time. You need a more effective way of judging the value of each move.

 

12 hours ago, sjhalayka said:

I'm having trouble knowing when to stop the learning process

And this indicates that you need to study this aspect. Andrew Ng's online course explains how to observe how well a system is learning and how to adjust it to improve.

I admit that I didn't look at the game link above, but I'm really having a hard time imagining how a NN (of any flavor) would be a good fit here. If you want to learn NNs, pick another game. If you want to do the AI for this game, pick another technique.

Dave Mark - President and Lead Designer of Intrinsic Algorithm LLC
Professional consultant on game AI, mathematical modeling, simulation modeling
Co-founder and 10 year advisor of the GDC AI Summit
Author of the book, Behavioral Mathematics for Game AI
Blogs I write:
IA News - What's happening at IA | IA on AI - AI news and notes | Post-Play'em - Observations on AI of games I play

"Reducing the world to mathematical equations!"

11 hours ago, Kylotan said:

This book is 24 years old. The state of the art has advanced SIGNIFICANTLY since then, not to mention the computing power available. Do not use that book as an example of what to do today.

In particular, genetic optimisation of neural networks made sense in the 90s; it mostly doesn't, now.

Do you have any recommendations on books to get?

 

11 hours ago, Kylotan said:

This sounds like you don't really understand what the inputs and outputs of the net should be. If you expect to train the network by running entire games and then scoring it with a Win or a Lose, you're throwing away tons of useful information and wasting time. You need a more effective way of judging the value of each move.

Well, I'm pretty sure that the inputs and outputs will be states, with the input/output pair counted as a state change. When a move is made, the AI gives a proposed state change. If that proposed state change is invalid, then a state change is picked pseudorandomly from a list of known valid state changes. When a game is won, all of the state changes picked during the game are ran through the back propagation process. When a game is lost, all of the states changes are abandoned.

Do you know of a more efficient way?

 

11 hours ago, Kylotan said:

And this indicates that you need to study this aspect. Andrew Ng's online course explains how to observe how well a system is learning and how to adjust it to improve.

OK.

5 hours ago, IADaveMark said:

I admit that I didn't look at the game link above, but I'm really having a hard time imagining how a NN (of any flavor) would be a good fit here. If you want to learn NNs, pick another game. If you want to do the AI for this game, pick another technique.

That's ok. What games would work with a NN? What kind of AI might work better for a poker game?

I believe that I understand what you mean by intractable.

OK... so you are really, really sold on doing something with a NN? That's your entire focus here?

NNs are largely only good at pattern matching (as you seem to understand). That is, "given this exact set of inputs, what's the best output?" 

Dave Mark - President and Lead Designer of Intrinsic Algorithm LLC
Professional consultant on game AI, mathematical modeling, simulation modeling
Co-founder and 10 year advisor of the GDC AI Summit
Author of the book, Behavioral Mathematics for Game AI
Blogs I write:
IA News - What's happening at IA | IA on AI - AI news and notes | Post-Play'em - Observations on AI of games I play

"Reducing the world to mathematical equations!"

6 minutes ago, IADaveMark said:

OK... so you are really, really sold on doing something with a NN? That's your entire focus here?

NNs are largely only good at pattern matching (as you seem to understand). That is, "given this exact set of inputs, what's the best output?" 

I'm not really sold on a NN solution, definitely less so since this discussion. :)

I'm just going to do hard-coded AI, using if else statements and the like. I do appreciate you steering me away from using a NN, which doesn't seem to be the best solution.

I found an old copy of AI for Game Developers from O'Reilly.

If I were to use an ANN, I would also have to encode the game state. Not that I'm going to use an ANN.

Start here...

http://intrinsicalgorithm.com/IAonAI/2012/11/ai-architectures-a-culinary-guide-gdmag-article/

Dave Mark - President and Lead Designer of Intrinsic Algorithm LLC
Professional consultant on game AI, mathematical modeling, simulation modeling
Co-founder and 10 year advisor of the GDC AI Summit
Author of the book, Behavioral Mathematics for Game AI
Blogs I write:
IA News - What's happening at IA | IA on AI - AI news and notes | Post-Play'em - Observations on AI of games I play

"Reducing the world to mathematical equations!"

Thanks for the information. Now I'm hungry. :)

I found these two articles to be cautiously optimistic about the use of ANNs as a poker player:

https://mattmazur.com/2009/10/13/experimenting-with-a-neural-network-based-poker-bot/

https://www.pokernews.com/news/2017/03/ai-uses-neural-networks-for-poker-intuition-27301.htm

 

Note that what you're talking about is not really what most people consider "poker" - it's (from what I can tell) a hand-assembly game where hands are scored using poker rules.

This is actually a very important distinction.

Playing, say, Texas Hold 'Em with an ANN requires one specific set of game knowledge for the NN to be effective. Playing your game requires a very different set of game state and game awareness representation.

So your links are basically like finding a recipe for cake and deciding that the ingredients would make a good hamburger, because cake and hamburgers are both eaten. You can't cherry-pick one part of the system (NNs) and slap it into a disparate game and expect to get equally good results.

What people are trying to tell you in this thread is that you need to decide what you are doing. Either focus on NNs and apply them to a game they are suited for (i.e. NOT your game) or focus on your game and find a technique that works for it.

Wielder of the Sacred Wands
[Work - ArenaNet] [Epoch Language] [Scribblings]

This topic is closed to new replies.

Advertisement