Perfect Evaluation for Connect four.

Started by
40 comments, last by mandar9589 13 years, 11 months ago
The language I am using is java.(I know that this language is not used for game programing as c or c++,however when it comes to graphics and uploading on internet,with java it can be easily done.)

There are few questions which I want to ask you.

1. Why are java games slower in computing than games programmed in C/C++? is it that creation of objects in java takes more time, or is it the memory allocation?

2.I have programed this game using negamax algorithm. I added alpha beta pruning to it. Since the number of nodes were very high even after pruning, I gave it a beta cut-off by writing
if(alpha>beta) return alpha;

again after writing this, the nodes very very high in number.
so I replaced alpha>beta by alpha>=beta. this change made drastic decrease in number of nodes. Is this correct?

3.Right now I have deleted all the previous evaluations done,and writing new one. so the basic step I took was to write the winning conditions only, on depth of 14 it takes about 2.91 mins(This is too much of time,and want to reduce the time significantly.) to compute the first move on the empty board.
and about 15ms to calculate at depth of 4. So how can I calculate the time which will be required to search till depth of 42plys?, I can see there is exponential relation in this.

4.Since now my method has only one checking of whether its winning or not, it is computing very fast and more number of loops I add to this method, the more time it takes and this is undesireable. So even if I want to add all the configurations which will state whether its a odd threat or even threat, there would be atleast 16 more conditional statement in 4 loops and this will take most of the time and so my search depth has to compromise with the time available for search.and this is again undesireable.

So do you have any more ideas as to how to speed up the search of the game,without reducing the depth,one of them might be using hashtables along with zobrist keys ?



Advertisement
Quote:Original post by mandar9589
1. Why are java games slower in computing than games programmed in C/C++? is it that creation of objects in java takes more time, or is it the memory allocation?

I can think of several reasons. The abstraction that the virtual machine provides has a cost in performance, even if Java supporters tend to deny it.

It's probably the case that people that learn Java first develop habits that hurt performance in this type of program (although those habits might be good for other types of programs). For instance, I had been programming for 10 years before I heard the word "object", so I know how to program without them. If you go around creating objects all over the place, your performance will suffer (mostly because object creation in Java always implies dynamic memory allocation, if I am not mistaken). A fast alpha-beta searcher doesn't do any dynamic memory allocation.

Quote:
2.I have programed this game using negamax algorithm. I added alpha beta pruning to it. Since the number of nodes were very high even after pruning, I gave it a beta cut-off by writing *** Source Snippet Removed ***
again after writing this, the nodes very very high in number.
so I replaced alpha>beta by alpha>=beta. this change made drastic decrease in number of nodes. Is this correct?

Yes, that is correct. There will only be a huge difference if your evaluation often returns the same values.

Quote:3.Right now I have deleted all the previous evaluations done,and writing new one. so the basic step I took was to write the winning conditions only, on depth of 14 it takes about 2.91 mins(This is too much of time,and want to reduce the time significantly.) to compute the first move on the empty board.
and about 15ms to calculate at depth of 4. So how can I calculate the time which will be required to search till depth of 42plys?, I can see there is exponential relation in this.

Yes, there is an exponential involved. A search of depth 15 typically will take some constant factor of time more than a search of depth 14 (say, 2.5 times more time). The factor is what I call "effective branching factor", although I don't think that term is standard.

The effective branching factor depends on how well you program your search. In the worst case, you always explore the worst moves first and the best moves last, and then alpha-beta doesn't prune anything and your effective branching factor is around 7 (whatever is the true branching factor of the game tree, but let's say 7 for simplicity). If you had perfect ordering, the branching factor would be sqrt(7). But then again, perfect ordering requires knowing what the best move is, and if you know that you don't need to search. ;)

In practice, you should get close to the sqrt(7) limit. Actually, hash tables can bring the effective branching factor down even lower.

You should measure how long it takes your program to search depths 1, 2, 3... (until you get bored of waiting), and then plot the logarithm of the time spent. You'll probably get something close to a straight line, whose slope indicates the effective branching factor. This is a very useful thing to do if you are working on improving your move-ordering heuristics or your hash tables.

Quote:4.Since now my method has only one checking of whether its winning or not, it is computing very fast and more number of loops I add to this method, the more time it takes and this is undesireable. So even if I want to add all the configurations which will state whether its a odd threat or even threat, there would be atleast 16 more conditional statement in 4 loops and this will take most of the time and so my search depth has to compromise with the time available for search.and this is again undesireable.

Oh, the evaluation function that checks threats correctly will be much more complicated than that. When I wrote this with a friend many years ago, we ended up coding it in 1600 lines of assembly code. But a depth-8 search using this evaluation function will probably play much better than your depth-14 search with no knowledge of the game.

Quote:So do you have any more ideas as to how to speed up the search of the game,without reducing the depth,one of them might be using hashtables along with zobrist keys ?


The first thing is move ordering. Some form of "history heuristic" (look it up) will work well.

Hash tables also help, both because they eliminate searching repeated branches in some conditions and because you can store the most promising move to be explored first if you have to search this node again.

You should also use iterative deepening. Even if it sounds counter-intuitive, searching with depth 1, then 2, then 3, ... up to depth 14 is faster than searching depth 14 directly. Well, at least it is if your hash tables and move-order heuristics work well.

I think you should forget about perfect play for now and simply make a strong program. This is actually better practice if you want to eventually make programs for games that are too hard to fully solve (e.g. chess).
Ya I guess I have to develop my level of coding first rather than looking for perfect play.

I tried for transposition tables and tried to code them in java.
I generated a 64-bit random number,(The website which gave info on this was written for chess programing. So I don't know whether I have to use 64 bit TT, also I think that I have to use it because I found that even John Tromp used in his applet.). However, I am not able to initialize the array that I will use for my hash table(I don't know whether Hash tables in java are more powerful than arrays in this matter.)because I don't know the exact size that would be required.

What size of the table should I go for? also is there any mathematical formula regarding the same?
Transposition tables are one way to trade CPU time for memory consumption. Make them as large as your RAM can comfortably accommodate. Try 1GB if you have 2 or more of physical memory.

Yes, you can use 64-bit numbers for hash keys. John Tromp has a method to compute them that guarantees no collisions (it turns out there are fewer than 2^64 positions in connect-4). It goes something like this: Use 7 bits for each column, one per square in it, plus one extra that you can imagine as being on top of the board. Now use a 1 the first empty square in the column, a 0 for other empty squares, a 1 for player 1's disks and a 0 for player 2's disks. This encodes the position completely in 49 bits.

If you want to use Zobrist keys (like in chess), that works too.
I started implementing the TT,
I broke it into several steps.
1. writing a method which deals with generation random numbers.
2. Making a key for the table.

I have programed the first two and now I am facing trouble in programing the real hash table.

First of all, I will show what I did and then I will put up my query.

public static long Random()      {        long rand = random.nextLong();        return rand ^ (rand << 15) ^ (rand << 30) ^                (rand << 45) ^ (rand << 60);      }


The above method deals with generation of 64-bit random number.

[source lang= "java"] public static long Zobrist_Key(int board[][])    {        long key=0;        int r=0;        int c=0;        int color;        long zobrist[][][]=new long[rows][cols][2];        for( r=0;r<rows;r++)            for( c=0;c<cols;c++)                for(int piece=0;piece<2;piece++)                    zobrist[r][c][piece]=Random();        for(r=0;r<rows;r++)            for(c=0;c<cols;c++)            {         color=board[r][c];         key^=zobrist[r][c];<br>            }<br>        <span class="cpp-keyword">return</span> key;<br>    }<br><br><br></pre></div><!–ENDSCRIPT–><br><br>This method deals with generation of key.<br><br>This is the pseudocode which I am referring to:<br><!–STARTSCRIPT–><!–source lang="c"–><div class="source"><pre><span class="cpp-keyword">int</span> ProbeHash(<span class="cpp-keyword">int</span> depth, <span class="cpp-keyword">int</span> alpha, <span class="cpp-keyword">int</span> beta)<br><br>{<br><br>    HASHE * phashe = &amp;hash_table[ZobristKey() % TableSize()];<br><br> <br><br>    <span class="cpp-keyword">if</span> (phashe-&gt;key == ZobristKey()) {<br><br>        <span class="cpp-keyword">if</span> (phashe-&gt;depth &gt;= depth) {<br><br>            <span class="cpp-keyword">if</span> (phashe-&gt;flags == hashfEXACT)<br><br>                <span class="cpp-keyword">return</span> phashe-&gt;val;<br><br>            <span class="cpp-keyword">if</span> ((phashe-&gt;flags == hashfALPHA) &amp;&amp;<br><br>                (phashe-&gt;val &lt;= alpha))<br><br>                <span class="cpp-keyword">return</span> alpha;<br>            <span class="cpp-keyword">if</span> ((phashe-&gt;flags == hashfBETA) &amp;&amp;<br><br>                (phashe-&gt;val &gt;= beta))<br><br>                <span class="cpp-keyword">return</span> beta;<br><br>        }<br><br>        RememberBestMove();<br><br>    }<br><br>    <span class="cpp-keyword">return</span> valUNKNOWN;<br><br>}<br><br> <br><br><span class="cpp-keyword">void</span> RecordHash(<span class="cpp-keyword">int</span> depth, <span class="cpp-keyword">int</span> val, <span class="cpp-keyword">int</span> hashf)<br><br>{<br><br>    HASHE * phashe = &amp;hash_table[ZobristKey() % TableSize()];<br><br> <br><br>    phashe-&gt;key = ZobristKey();<br><br>    phashe-&gt;best = BestMove();<br><br>    phashe-&gt;val = val;<br><br>    phashe-&gt;hashf = hashf;<br><br>    phashe-&gt;depth = depth;<br>}<br><br><br></pre></div><!–ENDSCRIPT–><br><br><br><!–STARTSCRIPT–><!–source lang="c"–><div class="source"><pre><span class="cpp-directive">#define</span>    hashfEXACT   <span class="cpp-number">0</span><br><br><span class="cpp-directive">#define</span>    hashfALPHA   <span class="cpp-number">1</span><br><br><span class="cpp-directive">#define</span>    hashfBETA    <span class="cpp-number">2</span><br><br> <br><br><span class="cpp-keyword">typedef</span> <span class="cpp-keyword">struct</span> tagHASHE {<br><br>    U64 key;<br><br>    <span class="cpp-keyword">int</span> depth;<br><br>    <span class="cpp-keyword">int</span> flags;<br><br>    <span class="cpp-keyword">int</span> value;<br><br>    MOVE best;<br><br>}   HASHE;<br><br><br></pre></div><!–ENDSCRIPT–><br><br>No I don't understand few things in this,<br><br>1] what does "HASHE * phashe = &hash_table[ZobristKey() % TableSize()];"  exactly means is it the '*' pointer or multiplication sign, and if it is a pointer, how to deal with it in java? also same thing about "&hash_table"<br>I guess it is the 1-d array formed by randomly generated array be operations performed &#111;n zobrist key and Table size and how to deal with "&" pointer? I googled for replacing "& in C which is something similar with java". also how to initialize "phase" ?<br><br><br>2] there is no pseudo code provided for "BestMove()" and " RememberBestMove()"<br><br>3] What should be the value of "valUNKNOWN;" variable in order to initialize this<br><br>Here is the link I am referring to <br><a href='http://web.archive.org/web/20040420021621/brucemo.com/compchess/programming/hashing.htm'>http://web.archive.org/web/20040420021621/brucemo.com/compchess/programming/hashing.htm</a><br>Please let me know if I am wrong in this case, I am programing TT for the first time.
Don't take this the wrong way, but I am starting to believe that you should try to fight your way through these things on your own. You'll learn a lot more that way than if someone holds your hand through the process.

Your code to generate 64-bit random numbers actually just takes a pattern of 15 bits and repeats it. That's not very random. If you are asked to generate a random 10-digit number, you probably wouldn't pick a random digit and then say that 4444444444 is your answer: You need to pick each digit separately.

Your code to compute the Zobrist key shouldn't generate a new table each time. In fact, what makes using Zobrist keys practical is that you can cheaply update their value as you make changes to the board. So make the Zobrist key part of the board description, and make sure that when you add or remove a disk it is updated correctly.

Bruce Moreland's code is using common C/C++ notation. Yes, the `*' is a pointer, and the `&' means "address of". Sorry, I don't know Java.

Hmm.. I think I have to take my own time with this thing.
any ways I didn't understand why you said that it repeats a pattern of 15 digits.

Here is what I got after printing the Zobrist key table.

316675903089649482
4182044110946497527
8679767581798476459
8855815849626324898
3518128907778205690
7408368457306139219
2541910641945211326
1330732081786807354
-4786410104812039334
-3539844555422608188
5153079562565024164
896840132128749395
-8448075625770118985
9191389141439803184
8767227441167592200
-7959684096243951936
-6824008056587565641
-1080299956079260171
-4594278649836306905
-3694024200017839097
-7668757661806015280
-342349832775026588
1161613343615965523
4895476057721425637
6237447854382709710
2453178027714057025
8679656353642823537
1225031101019518918
-4666244494015235503
3147982707560520517
-9082230669594423685
8383571467081531981
-7479884157709440948
-2694274935071035492
2413211691022421504
5670075944325812178
3945919730021029000
-2333830053532767128
1170620168546520479
-8503006286477764129
5168129314257060638
8548437632252761404
3005322102905754511
-7898656530010602317
-3303232157651303423
-4678317885382340703
-7012451636247569638
-689354272021937729
6614047025257945458
-3496125424209662572
6918107347829187358
5042924591036645931
-6561271854487048566
-6656019709916018682
2027087705067998347
6492521752825919826
3964584853325651379
-9185507933098792613
-6535061969217599257
-7820585283447017663
-7093445561254115537
5291390924821743596
-7497377898076677939
3150967723600683836
-1502313211433109616
6884721904849014917
2121988517770041961
-4796603394943156678
3004738569651396158
-5388458876790210329
-8931755014229497858
-6700449956692109980
6062014883500332021
4747269958332355143
-3232583153369332128
8168737574929661082
3202243026128613700
-5632740327420513331
-653352630854573190
-3707981595352487698
8565720777665279246
-6620282677889608426
4899598924683338954
-3540588538895244506

I am unable to see what is repeating.
Hmmm... No, nothing seems to repeat there. Why did you do all those shifts and xors with the random number? I thought your pseudo-random number generator was giving you only 15 bits and that's why you were fiddling with the result.

If your random-number generator already gives you 64 bits, there's no reason to do any of that.
I have tried to speed up the algorithm my providing history of highest value search. How ever the speed is going on decreasing. I have tried to sort all moves in descending order according to its values. I read this in online thesis written Quain.

 for (int x = 0; x < (count / 2); x++)          {            temp = list2[x];            list2[x] = list2[(count - x) / 2];            list2[(count - x) / 2] = temp;          }// Keep middle column as main priority from start. If there is change in value, change the priority from middle column to higher priority column.        for (int a = vals.length - 1; a >= 1; --a)          {            int currentMax = vals[0];            int maxIndex = 0;            int currentMaxNum = 0;            int maxValue = list2[0];            for (int b = 1; b <= a; b++)              {                if (currentMax > list2)                  {                    currentMax = vals;                    maxIndex = b;                    maxValue = list2;                    currentMaxNum = b;                  }              }            if (maxIndex != a)              {                vals[maxIndex] = vals[a];                vals[a] = currentMax;                list2[maxIndex] = list2[a];                list2[a] = maxValue;              }          }return list2;


i guess this is what is called as history heuristics.Thinking time is higher as compared to no history heuristics. Just tell me whether I am implementing it in right way or not , not asking for debugging the code, but to give pointers as to what is going wrong.
From your description, it is not clear to me where you get the values to sort by.

There are many different variations one could try around the basic idea of history heuristic. Of the top of my head:
* Add something to the heuristic whenever a move causes a beta cut-off, or do it after every search, adding to the best move found.
* Add 1, or depth, or depth^2, or 2^depth.
* Have 2 tables of 42 value (how good is move <X> for player <P>), or have 2 tables of 42*42 values (how good is move <X> in response to move <Y> for player <P>).
* Subtract something from the values of the moves that were tried before the beta cut-off but did not generate a cut, or never subtract anything.
* Etc.

The fact that the first thing you tried didn't work doesn't mean that nothing will work.

The work of developing an alpha-beta searcher involves trying many ideas, and there is no guarantee that a particular idea will be an improvement in your program, so you need to set up good methods to test your progress.

This topic is closed to new replies.

Advertisement