Archived

This topic is now archived and is closed to further replies.

myfirstgame

Neural Nets

Recommended Posts

myfirstgame    122
Hello,Every Body: I have a project that is online rpg game. My responsibility is AI. I want to implement the AI in each enemy during the battle. I want each enemy can have self learning during the battle. So, I want to use finite state machine and neural nets technology to implememnt it. But I don''t know how to do. Although I have read several web site about this technology. But it just give me concept. I don''t really know how to apply them,can you help me,if possilbe,give me some example or code to describe how to do it,thx

Share this post


Link to post
Share on other sites
Tricron2    122
Well a finite state machine is pretty simple...
Basically you just all of your fuctions have
if state==1
do this
if state==2
do this
if state==3
do this

Although its ussually better set up with classes and such.
As for Neural nets... I''m not entirely sure, hes what I''ve got about them.
Basically, you take all the data you can gleam from an event, and assuming you had X independent data results (from whatever)
you would plot that in X dimensional space. Then you can predict outcomes by comparing them to the nearest outcome in that space.
That may be completely off... I don''t really know.
but heres an example.
In voice dictation, to simulate learning, the record the pitch, tombre, vibrato, volume, and length and whatever else about each letter you make. Then it plots it in N-dimensional space. Then when you say something else, it plots that and finds the closest entry, and plots that letter from that entry.

Again, about neural nets I may be completely off... I''m interested in knowing if I''m at all correct.

Share this post


Link to post
Share on other sites
edotorpedo    198

Well, you''re right about the voice dictation. But what you have to realise is, that neural nets GENERALIZE, and they don''t provide a solution out of nothing. You have to train them with input and target data. Therefore I don''t think you can use them for overall tactics, unless you have some sample-data.

You could however use neural nets for targeting, economics and production systems etc...

About the technology, I have a DLL which can create *any* kind of neural net and train it. You can easily modify it to have multiple nets. I''ll be posting it somewhere soon.

Edo

Share this post


Link to post
Share on other sites
Mattr    122
Hi,

As edotorpedo said, NN''s (Neural Nets) need to be tought. To try and put it simply. You start off with a number of ''Neurodes'', effectively, these are just nodes in the network, that form the main structure. In between all of your Nodes, you have connections, with varying weights, and the Nodes can, if you want, have some bias also.

So what we do when we want to create a Neural Network is fisrt design the Network architecture, i.e. inputs, outputs, and hidden layers. So, we''ll use an example:

I made a game that used Neural Nets to play pong, so the inputs are: The x-coordinate of the ball.
The y-coordinate of the ball.
The direction of the ball.
We therefore have three inputs, the output is the desired x-coordinate of the bat. So we design the Network. (Incidentally, it doesn''t matter how the whole network is composed, it can take some fiddling to find a good one, but I''m led to believe that it should constrict as it goes from Input to Output).

so, the architechture may be:

x
x
x x
x
x
But each node needs to have a weighted connection to the other nodes, of course this can be zero, but it needs to be a fraction, either positive or negative, less than +/- 1. We randomly assign these values at first.

Now you need to make it learn, so what you do is give it a training set, with a set of input values, and their corresponding output values, then use some training algorithm, I used Back Propogation (which I can''t remember the algorithm for, and can''t be bothered to find my notes either, sorry).

This algorithm will pass through the network, and apply the inputs, and get the corresponding output. It will compare this with the expected output, and get an error value. It then updates the weights on each connection within the network in order to decrease the error, what we call ''reaching a lower point in weight-error space''.

So what you have after maybe one million epochs (learning iterations) is a network that can perform a certain function. So in my example, it could track the position of the ball on the screen. But try and make the same network sort three numbers, and it''d have no chance!!! That''s the thing, once you have a NN, it is specific to the task it has been trained in, and it interpolates fairly well, but if you try and extrapolate data from it, it tends not to do so well. But the thing is, I can save my network, and it does my required function, so next time I want to use it, I don''t have to train it again. Although, you could make it learn as it goes, but be careful! As you increase the size of the training data set, the amount of data that the NN is trying to correlate could possibly become more sparse, and the performance of the NN could degrade, that''s the problem I had with my pong game when the NN learnt from mistakes, it did well other than that.

By the way, that''s just an intro to Multi-Layer Perceptrons, a fairly common NN, but there are other types that may suit the application better, and of course, there are other learning algorithms. They can be quite complex to make a general, multi-purpose one.

I have a Neural Network Component that I''ve developed, it uses the above techniques, and allows you to generate any Network architecture, and training set in simple text file format. It''s done in Delphi.

Share this post


Link to post
Share on other sites
EvilCrap    134
how big are your battles? do they have 5 guys, 50 guys, or 500 guy?

you need to go about thinking how ai effects situations.
the ai to control 5 guys is different than ai to control 50 guys or 500 guys... which is differnt from ai that controls 1 guy.

for the most part, you need several layers of ai. you need a strategic ai, that can examine all units, and you need tactical ai, or ai controled by individual units.

although i realize that you want to use a neural net, i dont think you know how yet, do a pong or something first.

but, if you want to make this game, dont use neural nets, unless you have ALOT of time to develop them properly.

a better bet is to use genetic algorithms. this way, you can have a base ai that you can actually play with, and it will get harder to beat each game.

-if its on a computer, its a finite state machine.

Share this post


Link to post
Share on other sites
TerranFury    142
You don''t need neural nets. I''m not even sure how you''d go about applying NNs to something like this.

Just use an FSM with some random probabilities. Tweak it to taste and call it a day.

Share this post


Link to post
Share on other sites
fup    463
I think a lot of you need to learn a little more about neural nets. ;0)

Quoting from the neural net FAQ:

"In principle, NNs can compute any computable function, i.e., they can do everything a normal digital computer can do (Valiant, 1988; Siegelmann and Sontag, 1999; Orponen, 2000; Sima and Orponen, 2001), or perhaps even more, under some assumptions of doubtful practicality (see Siegelmann, 1998, but also Hadley, 1999). "

for further information see:

ftp://ftp.sas.com/pub/neural/FAQ.html#A_cando

You *can* use an ANN for this type of AI but it is inadvisable in this example unless you are prepared to devote a *lot* of time experimenting. A FSM is much easier to program and will give you far more control and acceptable performance.

Check out my website to see how you can use neural nets as a simple control system. You can find it here:

http://www.btinternet.com/~fup/Stimulate.html

(you need flash installed for the home page)

If you go on to do something interesting after reading my tutorial please let me know.

Share this post


Link to post
Share on other sites
TM    122
I was wondering about the applicability of NN to games... does anyone know a game which has succesfully used NN??

Share this post


Link to post
Share on other sites
alexjc    457
Collin McRae Rally 2, Black & White, Creatures

I think that''s about it! There are also three experimental hobbyist bots using NN, including mine...

Share this post


Link to post
Share on other sites
TerranFury    142
Some NNs are pattern classifiers. Take a series of inputs, and return an output that describes the inputs. EG: For an OCR program, take 100 inputs (for a 10 by 10 pixel grid) and ouput a number between 0 and 9 to describe the number portrayed in that square of pixels.

Some NNs are function estimators. Input one number and it spits out another. You could, using some simple physics formulae, create a function that, given the firing angle of a cannon of fixed muzzle velocity, returned the distace away the shell would land (assuming a flat plane). Or you could give an NN angles as inputs and actual landing spots as training values for a couple thousand epochs until it eventually, given an angle, outputs a number very close to the one that comes from your physics formula. Remember, they're function estimators .

Other NNs take several inputs (an input vector) and respond with several outputs (an output vector). This is similar to the first example I gave; it just classifies a pattern in terms of another pattern.

Think for a minute about whether any of the above situations accurately describe your NPCs. I doubt they do. For one thing, you have no training data. You have no known goals you want them to reach; all you want is sort-of-human-like behavior. Then, like most of the above posters, I feel that the best way to do this is to design the behavior yourself, using an FSM.



Edited by - TerranFury on February 5, 2002 5:16:57 PM

Share this post


Link to post
Share on other sites
fup    463
Alex: I''m interested in learning more about your bot project. Do you mean a bot for a FPS type game? If not, what sort of bot?

If it is for a FPS would you be kind enough to tell me a little about it as this application of ANNs has interested me for some time?

thanks

ps. maybe we could continue this discussion on your message board, what d''ya think?

Share this post


Link to post
Share on other sites
TerranFury    142
I can''t think of any way to use traditional techniques like backprop since there is no training data. The only thing I can think of (which is, btw, very very slow) is to use a GA to evolve your ANNs.

Share this post


Link to post
Share on other sites
fup    463
Training an ANN by evolutionary methods has been consistently proven to be more efficient than backprop.

It all depends how you go about it. One of the most important things is to maintain diversity amongst the population otherwise you get premature convergence and you just end up with brute force way (mutation) of traversing the search space. Many techniques have been dreamed up for this including fitness sharing, speciazation and SANE to name a few.

And then there are the ways of actually evolving the network topology like Kenneth Owen Stanley''s wonderful NEAT technique.

My tutorial doesn''t go into these sort of optimization techniques because its aimed at complete beginners and I don''t want to confuse the issue. It''s hard enough getting your head around ANNs in the first place!

Share this post


Link to post
Share on other sites
Timkin    864
If you don''t have training data then you need to implement a reinforcement learning approach, rather than a supervised learning approach. There is so much literature out there on these fields that I am not going to list any references... just do a google search.

Cheers,

Timkin

Share this post


Link to post
Share on other sites
edotorpedo    198

Hey Aspirin,

I just *read* (my german is a bit rusty, but still I could understand most of it) your article on neural nets, and I was wondering if you somewhere have a translation in english. Because I think your article is GREAT!, and could really help all neural net developers out there, except most developers don''t speak or read german.

Keep up the good work (in english) !

Edo

Share this post


Link to post
Share on other sites