willhMember Since 03 Mar 2009
Offline Last Active Sep 19 2012 07:00 AM
- Group Members
- Active Posts 174
- Profile Views 2,166
- Submitted Links 0
- Member Title Member
- Age Age Unknown
- Birthday Birthday Unknown
Posted by willh on 19 September 2012 - 07:00 AM
Basically, feed it things like 'passed pawns', 'player captures', 'opponent captures', etc.. All of the things you would calculate as part of a regular evaluation function become inputs in to the network. The network would output a score that determine how good (or bad) a particular position is.
In theory it should work just fine. In practice though it is going to be difficult to provide training data.
Posted by willh on 16 June 2012 - 09:49 AM
Second to that is Microsoft Kinect. It detects people and estimates poses in real time, even against cluttered backgrounds. Very artificially intelligent.
Most of the best AI is in things you don't notice... Path estimation to reduce the appearance of lag, algorithms to ensure realistic poses and animations, navigation algorithms, etc...
Most video game NPCs use scripted behavior or simple hard coded behaviors. You will find the most interesting AI where a machine needs to interface with reality, but at that point it tends to become very transparent.
Some UAVs have pretty slick AI for target tracking and line-of-sight planning. Googles self driving car is another example.
Posted by willh on 21 May 2012 - 06:06 PM
I'm particularly interested in seeing you apply the GA optimization. Have you thought about keeping the top N strategies, so that maybe you could have different play styles that are of the same difficulty?
Keep up the great work!
Posted by willh on 18 May 2012 - 07:40 AM
Dave, he wants an online learning algorithm. NNs are well suited to the task. It takes a trivial amount of time to train/retrain an ANN and the whole thing can be automated. You're always complaining that NNs have no use in games, and now this gentleman has a plan to use one with legitimate reasons.
Blessman: you can cite this website and the post. It is peer reviewed, as everything gets reviewed and commented on by people who have some experience working within the field. There is a video on YouTube of reinforcement learning being used to teach an opponent to play the game "golden spaceships". If you look you can find it-- it's not an ANN, but the idea is the same.
Dave makes a good point about the use of ML techniques in general. For video games they are almost never the right choice. Hand coded rules are easier to adjust to produce the desired behavior and most games require simple agents.
Posted by willh on 03 May 2012 - 11:20 AM
It doesn't help that most people's way of thinking of neural networks is they're just magical artificial brains.
In fact Neural Networks are mostly bullshit, created to solve problems that you don't want to or are not intelligent enough to code up a proper solution to. In these cases you just accept approximate solutions and state that you want any inputs to be mapped to outputs that are known to arise from similar inputs.
TL;DR: NNs are way over-hyped.
Over hyped, yes. Bullpoo, no. Misunderstood, definately.
An ANN is an approximation of a function based on observations. So is a linear regression. So is a running average. So is almost all of statistics. Hardly bull poo.
I consistently see problems where an ANN outperforms a decision tree.
Posted by willh on 02 February 2012 - 10:57 PM
For example:' a joke is only a good joke when the punch line is unexpected. The joke teller was deceptive, normally a bad thing, but you laugh because it's not actually threatening. You laugh when you see someone get hurt (football to the nuts, etc..) but only so long as the injury isn't serious.
Posted by willh on 05 January 2012 - 10:19 AM
Sine is a bizarre thing to grasp because it's used in many different ways. I couldn't learn it in school and struggled to understand, but seemed to have no problem teaching it to myself when I wanted to know how to triangulate radio sources. You just need to find the right way of looking at it.
If youre trying to calculate distances or specifc angles, then Sine is the ratio of the lengths between two specific sides of a right angle triangle. Sine(angle) = opposite length/hypotenuse length. In other words, how angle, height, and overall distance are related.
If you're trying to make a smooth wave, then sine is a smooth periodic amplitude function. Try graphing y = sine(x) where x is a value between 0 and 6.28.
Of course they are the same thing, but maybe looking at it one or the other will help.
Posted by willh on 31 October 2011 - 07:49 AM
If there would be one machine learning thing that I would actively look into, it would be Decision Trees. Decision trees are brittle so can add variance and do well with little data. You could then augment them as Random forest or boosted trees. I think that would be where I would start.
A close cousin to bayes nets that might be useful to some types of games (arcade shooter, anything requiring movement tracking and prediction) would be a particle filter. Finally, a bared down Reinforcement learning algo may be good for long term play in a strategy or RPG. All of these would be very hard to get right and take a lot of time when a simple markov chain based model or even Finite state machine would have done just as good or better for much less work.
Trees are that little tool that nobody ever talks about. You can derive probability distributions from them, use them for clustering, regression, and classification. They can be built in near real time, and a human can understand their output (it's a decision tree!!!).
I've posted a few screen shots of regression and boosted regression trees. I've even offered to write a tutorial. They lack a cool name maybe... Or the math isnt complex enough to invoke the voodoo factor? Not even one thumbs up though... <sigh>
Posted by willh on 20 October 2011 - 11:49 AM
I still don't know what sort of "neural network" is being used as a race car controller. What are its inputs (range, speed, acceleration sensors) and outputs (steering, throttle, brakes)? Does it hold some internal state? Is it really a neural network, or thinking of it as a generic concise representation of a lookup table would be more appropriate? What constraints (e.g. symmetry between left and right) can be imposed on its structure?
Without assurance that good car controllers are possible, I'm strongly inclined to attribute any problems with cars that keep crashing around to fundamentally inadequate architecture (mainly ignoring important inputs), not to insufficient or unsophisticated training.
He specified the inputs: they are range to wall, and he has four of them. Range to wall is meaured in distance I'm assuming, and not 'time to collision'.
Good car controllers are definately possible. Below are some decent ones.
a. This one made by evolving a neural network (video)
b. A prettier one using Neuroevolution
c. Even prettier one
It's not really that complicated a problem when there are no other cars on the track. With enough time the neural network would just memorize the optimal path.
I can't speak for the OP, but one reason to use a neural network (MLP) is that they can approximate any function 'as is' without special coding considerations. Changing the inputs/outputs doesn't mean re-writing the neural network code.
He is definately missing some inputs if he only has 4 direction sensors. Speed of car would be an important one.
Posted by willh on 17 October 2011 - 06:45 PM
Support vector machine will not work, as he wants a continuos output. He could try support vector regression. Basically want you want to do is regression. NOTE: Avlaro, if you meant SVR my apologies.
Goggle "regression analysis" and you will find enough to occupy you for a life time. The best choice depends on your data.
Your problem is non linear I am assuming, otherwise simple weighted linear multivatiate regression would work.
- There is nothing wrong with neural networks provided you convert your inputs in to a suitable range. Input normalization is where most people fail with neural networks.
- Random forests will do what you want, but depending on the data, a neural network could be better.
- Boosted regreression trees will also work, but you will want to be familiar with regression trees and boosting before you try it.
- Gaussian mixture regression reportedly works well on noisy data (data that obviously is missing at least one dimension of information), but the implementation isn't exactly straightforward.
- MARSplines are meant specifically for regression problems, I've never used them but from what I've read they seem like a good solution.
- Support vector regression is another excellent tool for regression problems. A lot like neural networks in terms of many training parameters to tweak.
- You could also try evolutationary programming. Slow training, but flexible.
All that said, and without seeing your data, I would recommend:
- Back propagation neural network
- a regression tree
I've attached some screen shots showing the differences between ANNs, CART, and Boosted regression on the same 'noisey' sine wave. The Mean Squared Error is in the top of each graph.
The ANN produces a smooth continous function but doesn't handle outliers very well.
Regression trees are able to handle data that isn't smooth and continous, but produces 'plateaus' where you see idential outputs regardless of the input.
Boosted regression gives you the best of both worlds-- smooth outputs, while handling outliers well. The above model is actually quite weak because I only boosted a few trees.
Posted by willh on 20 June 2011 - 04:09 PM
I've been interested in the potential of neural nets in video games for a while now. But I have been thinking. How good are they at action oriented games, and more specifically, fighting games? I have too worries: a) they suck at real-time gameplay, or b) they're ridiculously good at it and become inhumanly difficult. I guess I'm just wondering if anybody here has any idea of whether a NN is a good idea for AI in a fighting game.
Just want to echo what's already been said..
ANNs are useful because they are not problem specific. You can, in theory, throw any problem at an ANN, combined with a training algorithm of your choice, and get some sort of answer. It's not just pattern classification-- any kind of function approximation. .
ANNs are usually a poor choice for solving a problem because they are not problem specific. The word 'poor' here is subjective. You can solve almost anything using backpropagation if you fiddle around enough, and the answer will be almost as good (or as good) as the best tool for the job. This should surprise nobody since z = f(x*y) is kind of flexible.
Posted by willh on 20 June 2011 - 03:58 PM
But it seems to me like the requirements are somewhat contradictory. If you know a lot about a problem, surely there's a better method than just randomly moving population of search points around and praying for emergent behavior? Are there any concrete examples where genetic algorithms are actually useful?
Avlaro kind of summed it up. I've used GA's before, with some success, but there have always been better alternatives and I was just 'playing'.
Their strength is that they are not problem specific, and in theory could be used to 'solve' just about any problem given enough computing time.
In reality this is mostely pointless. GAs, and ANNs, were dreamt up in a time when processing power (and raw data) was very limited. These days it's feasible to do a somewhat exhaustive search of a space using millions of data samples, so....
As far as games go, I can think of a few uses, but so far haven't seen anyone use them.
Posted by willh on 15 March 2011 - 08:31 AM
Algorithmically, solving chess is fairly simple. It's just a matter of throwing enough horsepower at it. Deep Blue was not impressive in its chess prowess nearly as much as it was impressive in its computing power. The same can be said of Watson on Jeopardy.
I do not agree. Watson (Deep QA) is a very different beast and is not just a game tree. Watson is more 'AI' than Deep Blue.
There are, as far as a I know, no equivilants to WATSON available on the desktop running at a much slower speed.
Posted by willh on 27 January 2011 - 11:56 AM
There is a good reason why nobody is talking about ANNs these days. It's 2011; Support Vector Machines are what's in. They do a better job at generalization, work for both classification and regression, and are way easier to tune than an ANN. That's not to say they are useless, it's just that 99% of the time they are the wrong tool for the job-- ESPECIALLY when you are hard coding the topology.
My advice to you is to hand code a multilayer perceptron using a LINEAR activation function (i.e. simple threshold) to solve XOR. Then do the same thing using a non-linear activation function.