Neural Network Learning??
quote:Original post by TerranFury
Baked. It''s been done. But it''s not very interesting, n''est-ce pas?
Why couldn''t the neural network continue to learn once in the field? Would that be interesting enough?
-Predictor
http://will.dwinnell.com
As for how ANN''s are applied in the chess engines that apply them..
Such chess engines are made up of three basic parts..
1) a Search() function which incorporates a minimax search such as alphabeta, MTD(f), etc.. and returns the best move and the minimax score found within the search depth.
2) a static Eval() which evaluates a static chess position (usualy at the end nodes of Search()) and returns a value representing which side is winning or losing at that node. This is the function they want to replace with an ANN.
3) The self-learning algorithm
Now consider a position P.. If we Search(P, Depth) to some specific depth we will get a minimax value V which is a better evaluation than we would get if we skipped the search and went straight to the eval.
The goal of ANN''s in these engines is to learn a new eval (I''ll call it NNEval(P)) to more closely match Search(P, Depth). If you can train NNEval(P) to always match Search(P, 6) for example, then NNEval() could be said to be equal to a Depth 6 minimax search.
Now replace the original Eval() with NNEval() in your Search(P, 6) and you have an ''effective'' depth of 12.
In practice you will not find a NNEval() such as this but the idea is a sound one. Most chess engines have a rather stupid end node evaluation and rely on great search depths to find the minimax move. A smart eval ''effively'' extends the search at the cost of taking more time to evaluate nodes which means fewer actual nodes searched.
If NNEval() ''effectively'' extends the search more than is lost due to the end node evaluation taking longer, then its a better alternative than not using it.
It should be pointed out that the strongest chess engines do NOT use ANN''s or any other ''learning'' algorithms in regards to their searching ability. Most top engines do some simple opening book learning but that is the extent of it.
There could be any number of reasons for this.. perhaps it is computationally infeasable to make an ANN based eval operate fast enough to cover the ground lost in the extra processing.
Or perhaps they just havent implimented their ANN''s in a well optimised fashion. A lot of computer chess literature is based on techniques to make the engine a computational monster. The tricks used to make these engines faster are very extensive - one of the current best engines, ''Fritz'', will search millions of nodes per second on todays single processor desktops computers and completely obliterates chess engines that use ANN''s.
For other games, ANN''s have proven very competitive to non-ANN implimentations.. The current computer Backgammon champion uses ANN''s and the best Go playing computers also use ANN''s.
Such chess engines are made up of three basic parts..
1) a Search() function which incorporates a minimax search such as alphabeta, MTD(f), etc.. and returns the best move and the minimax score found within the search depth.
2) a static Eval() which evaluates a static chess position (usualy at the end nodes of Search()) and returns a value representing which side is winning or losing at that node. This is the function they want to replace with an ANN.
3) The self-learning algorithm
Now consider a position P.. If we Search(P, Depth) to some specific depth we will get a minimax value V which is a better evaluation than we would get if we skipped the search and went straight to the eval.
The goal of ANN''s in these engines is to learn a new eval (I''ll call it NNEval(P)) to more closely match Search(P, Depth). If you can train NNEval(P) to always match Search(P, 6) for example, then NNEval() could be said to be equal to a Depth 6 minimax search.
Now replace the original Eval() with NNEval() in your Search(P, 6) and you have an ''effective'' depth of 12.
In practice you will not find a NNEval() such as this but the idea is a sound one. Most chess engines have a rather stupid end node evaluation and rely on great search depths to find the minimax move. A smart eval ''effively'' extends the search at the cost of taking more time to evaluate nodes which means fewer actual nodes searched.
If NNEval() ''effectively'' extends the search more than is lost due to the end node evaluation taking longer, then its a better alternative than not using it.
It should be pointed out that the strongest chess engines do NOT use ANN''s or any other ''learning'' algorithms in regards to their searching ability. Most top engines do some simple opening book learning but that is the extent of it.
There could be any number of reasons for this.. perhaps it is computationally infeasable to make an ANN based eval operate fast enough to cover the ground lost in the extra processing.
Or perhaps they just havent implimented their ANN''s in a well optimised fashion. A lot of computer chess literature is based on techniques to make the engine a computational monster. The tricks used to make these engines faster are very extensive - one of the current best engines, ''Fritz'', will search millions of nodes per second on todays single processor desktops computers and completely obliterates chess engines that use ANN''s.
For other games, ANN''s have proven very competitive to non-ANN implimentations.. The current computer Backgammon champion uses ANN''s and the best Go playing computers also use ANN''s.
go and backgammon use ANNs because ANNs are used primarily for detecting and evaluating patterns... this is especially necessary in GO because searching is such a stupid idea
Disclaimer: i'm not an expert
as far as i see it, i would not use ANN for decision system but for a representation system, a ANN would help an AI to recognize context and narrow the field of the schearch by eliminate by "experiance" some move, the ANN would have to learn online, then you must find a fct of evaluation of error wich would fct as a retroregulation, something like seeing the outcome of an action being bad or good
such an ann would not have only input from the board but also from the actual state of the decision system, the previous context, in order the ANN make inference between is own action and the way the game evolve, and then outputting a "temperature" of action, that would allow the system to "feel" if a move is good or bad
actually the system would have to learn, but i didn't remove the search ability of minimax fct, best, it enhance it through time
my favorite use of ann as i have toy with them is more for ginving evaluation hint
>>>>>>>>>>>>>>>
be good
be evil
but do it WELL
>>>>>>>>>>>>>>>
[edited by - neoshaman on March 24, 2004 7:37:46 AM]
as far as i see it, i would not use ANN for decision system but for a representation system, a ANN would help an AI to recognize context and narrow the field of the schearch by eliminate by "experiance" some move, the ANN would have to learn online, then you must find a fct of evaluation of error wich would fct as a retroregulation, something like seeing the outcome of an action being bad or good
such an ann would not have only input from the board but also from the actual state of the decision system, the previous context, in order the ANN make inference between is own action and the way the game evolve, and then outputting a "temperature" of action, that would allow the system to "feel" if a move is good or bad
actually the system would have to learn, but i didn't remove the search ability of minimax fct, best, it enhance it through time
my favorite use of ann as i have toy with them is more for ginving evaluation hint
>>>>>>>>>>>>>>>
be good
be evil
but do it WELL
>>>>>>>>>>>>>>>
[edited by - neoshaman on March 24, 2004 7:37:46 AM]
I am also not an expert...
Say you have an ANN, you can use positive/negitive reinforcement (meighting manipulation?) to retrain it. (encouraging/discouraging behaviours)
You teach it by the use of a simple Minimax (depth 1)
If its move is better then what it usually does, reward it (positive reinforcement), if it is worse, punish it (negitive reinforcement). if the same, reward it up to a point (to encourage future expantion, but allow the curent value to be learnt).
Game selling point: The AI learns to play off you *Learning its own style*, or you can get it to learn by itself!!
Perhaps have the rewards/punishments in steps, and allow the user to ''vote'' on how good the AI''s move was?
Say you have an ANN, you can use positive/negitive reinforcement (meighting manipulation?) to retrain it. (encouraging/discouraging behaviours)
You teach it by the use of a simple Minimax (depth 1)
If its move is better then what it usually does, reward it (positive reinforcement), if it is worse, punish it (negitive reinforcement). if the same, reward it up to a point (to encourage future expantion, but allow the curent value to be learnt).
Game selling point: The AI learns to play off you *Learning its own style*, or you can get it to learn by itself!!
Perhaps have the rewards/punishments in steps, and allow the user to ''vote'' on how good the AI''s move was?
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement