Jump to content

  • Log In with Google      Sign In   
  • Create Account


Efficient Implementation of Monte Carlo Player for Card Game


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
10 replies to this topic

#1 Alistair Sheehy Hutton   Members   -  Reputation: 133

Like
0Likes
Like

Posted 02 July 2013 - 03:31 PM

I've recently been looking at Briscola, which is a traditional trick taking card game.  I thought it would be relitaavely easy to produce Ai players for it, what with only ever having possible 3 moves a turn and a fixed number of turns to complete the game.  I decided I would knock up a quick Monte Carlo player to see what it played like.  The trouble is while I wrote it pretty quickly the AI is so slow iterating through games that I feel I must be overlooking some fundamentals of appropriate data structures to use to model the deck of cards.

 

Are there any good code examples of Monte Carlo based AIs for trad card games out there that I could learn from?



Sponsor:

#2 LorenzoGatti   Crossbones+   -  Reputation: 2315

Like
0Likes
Like

Posted 05 July 2013 - 01:41 AM

"Iterating through games"? Didn't you profile the program and/or try different problem sizes to measure which parts of your training are slow? For example, variant games with 8 AI players and 5 turns should be faster than games with 2 AI players and 20 turns if you have unneeded overhead per turn, slower if you have unneeded overhead per player, and equally expensive if the cost of each of 40 AI player moves is the same in both cases.
 
Can you explain what you are doing in more detail? What state and inputs do your AI players have?
Produci, consuma, crepa

#3 Álvaro   Crossbones+   -  Reputation: 10836

Like
0Likes
Like

Posted 05 July 2013 - 03:03 PM

Use a profiler to figure out what parts of your code are slow, then you can try different implementations. If you can't figure out why some part is slow, perhaps you can describe to us how you implemented it, in case there is something obviously wrong.

One general rule is to avoid dynamic memory allocation. Depending on the language you are using, this might be tricky to do. But with a bit of discipline, you'll realize that you don't really need it.

#4 Alistair Sheehy Hutton   Members   -  Reputation: 133

Like
0Likes
Like

Posted 08 July 2013 - 01:04 AM

Thanks for your responses, would have replied sooner but had problem logging back into the site.

 

I had profiled my code before posting and had spotted a few obvious inefficiencies, for example due to stupidity on my part I was shuffling the deck twice when performing a play through of the Monte Carlo player and still building up logging strings despite not printing them out.  Now that I have knocked out the (to me) obvious the largest amount of time by far is spent in random.shuffle().  Briscola is a trick taking game (normally 2 players) where players have a hand of three cards and  each round play 1 card then draw one card blind from the common deck.  So correct me if I'm wrong but each play through by the Monte Carlo player requires shuffling all the unseen cards to get a new ordering of the deck.  That's unavoidable correct?  I'm using Python so the shuffling algorithm is linear time and efficiently executed so I can't see much speed up there.

 

I think I might just have too many "speed of writing code vs speed of code execution" trade-offs in my Python programming style for this task - I should maybe do this in another language to force me to think about it differently.

 

PRE-EDIT:  Actually, writing this post has made me think critically about the code - I may have spotted another couple of gross inefficiencies.  I'll let you know how it goes.



#5 Álvaro   Crossbones+   -  Reputation: 10836

Like
0Likes
Like

Posted 08 July 2013 - 07:45 AM

Programming this in Python is already a problem because performance matters for Monte Carlo techniques. Do you know C++?



#6 LorenzoGatti   Crossbones+   -  Reputation: 2315

Like
0Likes
Like

Posted 08 July 2013 - 10:00 AM

You need a deck shuffle at most once per game, and it cannot plausibly cost more than the 1 or more random numbers per turn a "Monte Carlo" player presumably needs to choose a move (40 cards, 40 turns). While playing, the order of the cards remain fixed.  

 

You can even recycle the same shuffling of the deck many times with different random moves in each playthrough; depending on how do you evolve the AI players there should be effective ways to avoid overfitting.

What representation of game state, of available moves and of their own behaviour do your AI players employ? Reading between the lines I suspect horribly simplistic and inefficient methods, but with your sketchy descriptions I cannot be sure.


Produci, consuma, crepa

#7 Alistair Sheehy Hutton   Members   -  Reputation: 133

Like
0Likes
Like

Posted 08 July 2013 - 02:16 PM

Programming this in Python is already a problem because performance matters for Monte Carlo techniques. Do you know C++?

 

I've done C and C++, although given how long since I've last used them in anger I would hesitate to say that I am proficient in either.  This may be a good project to reacquaint myself with the language.



#8 AngleWyrm   Members   -  Reputation: 553

Like
0Likes
Like

Posted 31 July 2013 - 02:59 PM

So correct me if I'm wrong but each play through by the Monte Carlo player requires shuffling all the unseen cards to get a new ordering of the deck.

Why would you shuffle unseen cards more than once? They are already shuffled, so that the next card is unknown to the player. I see nothing more to be gained by shuffling it again.


Edited by AngleWyrm, 31 July 2013 - 03:09 PM.

--"I'm not at home right now, but" = lights on, but no ones home

#9 Álvaro   Crossbones+   -  Reputation: 10836

Like
0Likes
Like

Posted 31 July 2013 - 03:39 PM

Why would you shuffle unseen cards more than once?

You need to shuffle them once per game. If you are using Monte Carlo methods, you need to simulate many random games to accumulate statistics and you make decisions from the statistics.

#10 Alistair Sheehy Hutton   Members   -  Reputation: 133

Like
0Likes
Like

Posted 03 September 2013 - 03:11 AM

Finally having my first free weekend in a few months I'm almost done with my C implementation.  The data structures I've come up with aren't that fancy or complicated but are pretty efficient.  Once I got out of the mental dead end of the "deck of cards" actually behaving like a deck of cards with a 'top of deck' and the like I've been able to arrange the cards such that I need very few operations to save the current state and randomise the game state for the Monte Carlo playouts..

 

As you'd expect it's blazingly fast compared to my Python implementation, on the order of several hundred times speed up, possibly more.  Once I'm done with this I'll be interested to go back to the Python code and implement the same data structures in Python - once I'm no longer allocating and freeing memory all over the place I wonder how fast I can get the Python code to run.


Edited by Alistair Sheehy Hutton, 03 September 2013 - 06:13 AM.


#11 Angus Hollands   Members   -  Reputation: 597

Like
0Likes
Like

Posted 03 September 2013 - 12:05 PM

Before I write this, let me just state that I've made the (uninformed) decision to assume you're not well versed in Python optimisation. If you are, then fantastic, this post will be largely irrelevant. 

 

Python can be slower than C/++ in certain areas, but there are often times when the order of magnitude of the time it takes to complete a task is simply unnecessary. Python has a few good articles on the subject around the web, but things to consider: (most apply to loops)

 

  1. Make localised namespace lookups instead of global ones (eg. don't look up an attribute on x.y.z if x is outside the current namespace. Instead, store a reference to x.y as w, and lookup there. 
  2. Avoid tight loops, and where they're necessary, move as many operations outside the inner loops as possible. 
  3. Avoid throwing away objects after creating them, instead use a generator or other structures that don't create the entire object in one swoop.
  4. Make use of lazy condition evaluation to save making unnecessary calculations.

Using PyPy is also a very good way of improving performance, but be aware you should learn its ins-and-outs in order to best gain performance. Good luck.






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS