• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
Sign in to follow this  
Followers 0
sion5

Neural Network - Discussion

103 posts in this topic

Hi everyone, Im new to the forums although not new to the website. I'm just entering my final year of a Computer Games Dev Degree and looking for some advice. For my final year research project I aim to create an automated game agent using NN to control it. I want to explore the differences between using a genetic algorithm (GA) and backpropagation to get the synaptic weights(thats going to form the analysis). Has anyone made a NN using either method in a games environment? Are NN's good or bad in games? Anyone recommend any books/ websites to read about NN's in games? Does anyone have any advice to offer? Im looking to discuss the topic so I can form a foundation for my research. Contributors will be put into my project acknowledgement :-) Look forward to your comments
0

Share this post


Link to post
Share on other sites
Mentioning Neural Networks and Game AI, in this sub section...........

Run for the hills my friend run.. and never look back :).

Regards Wolfe
0

Share this post


Link to post
Share on other sites
And thus we see why game degrees from institutions are received with a raised eyebrow from the AI programmers.

Quote:
Original post by sion5
Hi everyone,

Im new to the forums although not new to the website. I'm just entering my final year of a Computer Games Dev Degree and looking for some advice.
Welcome to forums. However... be careful what you wish for.

Quote:
For my final year research project I aim to create an automated game agent using NN to control it. I want to explore the differences between using a genetic algorithm (GA) and backpropagation to get the synaptic weights(thats going to form the analysis).

A noble goal and an interesting project. However...

Quote:
Has anyone made a NN using either method in a games environment?

You would be hard pressed to find any commercial games that use NNs at all.
Quote:
Are NN's good or bad in games?

You can take my response above as either cause or effect. A) it could be that we don't have enough data because no one uses them for games, or B) it could be that, because they haven't been deemed good for games, no one uses them. The latter seems to be a lot more likely - especially considering how academia keeps trying to get us to use them in games.

Quote:
Anyone recommend any books/ websites to read about NN's in games?

I'm sure there are plenty - however, it depends on what your eventual goal is. If you are writing a paper or doing a project, you are OK. There are plenty of other people's projects/papers out there. If you are, as your previous questions imply, looking for practical usages, you are going to come up a little dry.

Quote:
Does anyone have any advice to offer?

Again, it depends on your goal. If you want to explore something for fun or as a curiosity, have at it. If you are trying to do something that is genuinely applicable to commercial games, or if you are hoping for a resume builder, choose a new topic.
0

Share this post


Link to post
Share on other sites
Decision trees seem like a really good use of an AI approach to games. They are very general, versatile, and can give really good results from what I've read. There's a thread here now which is asking about decision trees, and you can find a lot of information in the Game AI books.
0

Share this post


Link to post
Share on other sites
To be honest I was expecting a reply like this. As I mentioned, this is a research project, If at the end of it I can put up an arguement as much "against" as I can "for" using NN's in games, then I have achieved something.

"And thus we see why game degrees from institutions are received with a raised eyebrow from the AI programmers."

I find this comment a little rude. I have no doubts of the education I am receiving, we're an accredited course that work very closely with Blitz Games. I have also attended many games grads conferences and had a lot of good feedback from games companies.

Rant over... can I remind people of the XOR problem the single layer perceptron faced? becuase people hit a brick wall they dropped neural networks and said it couldnt be done! A few years later it took one person to investigate the problem to solve it (Multi layered perceptron).

Although im not trying to re-invent the wheel I am looking for advice on why NN's are so bad in games (if that is the case). Im looking for actual facts like when we implemented a NN we only got XXfps as apposed to the same problem with a decision tree at XXfps, or system resources increased by XX. Just saying that NN's dont work in games isn't really a solid reason, If you ask someone why they dont work your more likely to hear "because my friend said so" than because of an actual reason.

I guess I may be expecting a little too much from a forum but was hoping some AI programmer who has used them was floating around reading this somewhere.
0

Share this post


Link to post
Share on other sites
Hi,

I would not focus the use of NN only to games to see the weakness or strength of that approach
but look at other applications of NN. In autonomus driving they were used and the results looked promising but they arn't used that much now, find out way! (*exercise for the reader ;) *).
So the advice I'm offering you is: "Look around you and see where NNs are use and were not, that see if a game AI agent could use a tool like that."
0

Share this post


Link to post
Share on other sites
Thanks for the heads up. I have been investigating uses for NN's and really dont understand why their not used in games, I guess thats what im trying to establish. There has to be a technical reason why their not used! I know the game "creatures" used the technology and that was succesful.

Maybe I should start the discussion then by saying that the only problem I can see with NN's is that they are computationally expensive...BUT only for training! If this is done offline, what reason is there now for not using them?
0

Share this post


Link to post
Share on other sites
There are a few reasons for not using NNs in games. For one thing they're hard to debug. If a NN does something that you don't want in one part of the game, how do you change it? Its very hard to train it to deal with every situation that could come up in the game especially considering the different styles of individual players.

Another issue is the amount of time it takes to train. This wouldn't be an issue if you just had to train it once, but considering the number of feature changes the average game goes through from start to finish, not to mention tweaking and balancing issues you would have to retrain almost daily.
0

Share this post


Link to post
Share on other sites
I agree with debugging being a problem, but then isn't the success based on the method of learning you implement?

You also mention that you cant get a neural network to learn every scenario a game has to offer. Thats the whole point, you dont want to! The aim is to teach it enough so that it can "think" for itself and produce its own output. This is what creates the "unpredictable" AI aspect. If you tell it what to do in every scenario why call it AI? after all your trying to make the agent do things by itself not set its path for it.
0

Share this post


Link to post
Share on other sites
Quote:
Original post by sion5
"And thus we see why game degrees from institutions are received with a raised eyebrow from the AI programmers."

I find this comment a little rude. I have no doubts of the education I am receiving, we're an accredited course that work very closely with Blitz Games. I have also attended many games grads conferences and had a lot of good feedback from games companies.


Fair enough, yet they should also be making it clear that NNs are not typically used in games, and the reasons why.

Quote:
Although im not trying to re-invent the wheel I am looking for advice on why NN's are so bad in games (if that is the case). Im looking for actual facts like when we implemented a NN we only got XXfps as apposed to the same problem with a decision tree at XXfps, or system resources increased by XX.


It's not about performance, it's about semantics. Almost every problem can be better approached with another method, one which more closely takes into account the representation of the context and the previous knowledge available to the developer, not to mention often eliminating the need for training which is often time consuming, and which presumably needs to be redone if you find you need to adjust your network topology.

Quote:
You also mention that you cant get a neural network to learn every scenario a game has to offer. Thats the whole point, you dont want to! The aim is to teach it enough so that it can "think" for itself and produce its own output.


No, because that's just an academic aim. A practical or industrial aim would be to use whichever method copes with the situation best, so you'd typically go for a tool that doesn't require such contortions to get it to work in all cases. You could potentially spend forever adjusting inputs, outputs, and hidden layers to try and get your neural network doing something useful, with no guarantee of getting anything that is good enough to be playable, or you can pick a method that explicitly accounts for all the scenarios a developer can envisage and get it working more reliably.

Quote:
If you tell it what to do in every scenario why call it AI? after all your trying to make the agent do things by itself not set its path for it.


Again, the details of what you can really call AI is an academic concern. In industry the important thing is to get the agent doing what you want to meet the requirements of the game, not to get the agent thinking and perhaps doing something that you want in order to meet the requirements of some research into a method. If you're looking to real world implementations then they are invariably going to be driven by pragmatism.
0

Share this post


Link to post
Share on other sites
Thanks for that Kylotan, you raised some interesting points that I shall look into.

ID Merlin raised the point that Decision Trees would be a better alternative. Are these one size fits all approaches or do they have to be recreated for every game?

I may be fighting a loosing battle but i'm still a student(with no industry experience) and I CAN see potential for NN's.
0

Share this post


Link to post
Share on other sites
Quote:
Original post by Kylotan
A practical or industrial aim would be to use whichever method copes with the situation best, so you'd typically go for a tool that doesn't require such contortions to get it to work in all cases.

Let me begin by saying I agree with the OP's aim.

He said he's working on a final year research project. Key word is research. The whole point here is to try out something which is not commonly used in games (apparently), and see if he can improve on it. Perhaps find a fundamentally new way of using them. I admit I don't know that much about NNs, but I'm pretty sure people don't know EVERYTHING there is to know about them at this point. I'm sure innovation is still possible. And that doesn't happen if you don't attempt to improve on something that already exists, even if it was dismissed as an AI solution for games (perhaps prematurely, or maybe for good reasons).

Quote:
You could potentially spend forever adjusting inputs, outputs, and hidden layers to try and get your neural network doing something useful, with no guarantee of getting anything that is good enough to be playable, or you can pick a method that explicitly accounts for all the scenarios a developer can envisage and get it working more reliably.

And what happens in a scenario that the developer did not originally envisage?

IMO, this is where the AI can come closest to 'making or breaking' the game - those unexpected but possible scenarios.

Quote:
Again, the details of what you can really call AI is an academic concern. In industry the important thing is to get the agent doing what you want to meet the requirements of the game, not to get the agent thinking and perhaps doing something that you want in order to meet the requirements of some research into a method. If you're looking to real world implementations then they are invariably going to be driven by pragmatism.

Well, that certainly doesn't sit well with me, even if it is the reality. Personally, I wouldn't want to be a factory worker pumping out one game after anothre using nothing but previously known ideas and concepts, without the ability or any attempt to innovate.

You do bring up a good point, however. I now realize I wouldn't want to go into game development if it ends up involving nothing but spitting out cookie-cut, run-of-the-mill games.
0

Share this post


Link to post
Share on other sites
Spend a year sitting down the hall from a designer who wants total control of situational behaviors. That will sour you on NNs really quick. Black boxes are scary.

Also, beware "catastrophic unlearning".
0

Share this post


Link to post
Share on other sites
Thanks shurcool, I think you hit the nail on the head with what I was trying to explain about it being research.

Is the lack of NN applications in games a sign that there are no suitable problems?

Do game developers and designers lack the necessary skillset to apply NN?

Are NN missing some key technology that would allow more widespread adoption?

Personally I think its the old "stick with what you know best" routine! At the end of the day hardware is flying ahead now, their putting NN's right into the silicon these days.

I dont mean any offence here at all, but almost every post has made a statement, but not 1 person has backed it up with fact or proof. It may well be me (the academic) asking too much but I still stand with no proof that their a bad choice for games
0

Share this post


Link to post
Share on other sites
Quote:
Original post by sion5
Is the lack of NN applications in games a sign that there are no suitable problems?
It is a sign that none have been found to work to the complete satisfaction of a production (i.e. non-academic) environment for years now. Infer from that what you will. They could be used in spot areas here and there in very limited capacities - but they simply don't suit many of the problems that are faced by game agents.

Quote:
Do game developers and designers lack the necessary skillset to apply NN?
Designers often lack the necessary skillset to apply an IF/THEN statement. That's not their job. However, this is a discussion that has been happening for a while - how to bridge that gap better.

Quote:
Are NN missing some key technology that would allow more widespread adoption?
Fine tuned manual control with immediate, predictable feedback. That's the one that comes up the most.

Quote:
Personally I think its the old "stick with what you know best" routine!
I disagree - there are plenty of new approaches in every issue of AI Game Programming Wisdom, for example. And conferences. There's ground-breaking stuff happening all over the place. But despite seemingly every college student coming out trying to solve every problem with NNs, not a lot of it seems to stick. There has to be a reason for it.

Quote:
I dont mean any offence here at all, but almost every post has made a statement, but not 1 person has backed it up with fact or proof. It may well be me (the academic) asking too much but I still stand with no proof that their a bad choice for games

You are the one doing the research paper, not the people replying here. Exhibit A is the past 30 years of the industry. Also, one thing that you should have learned from science and logic classes is that proving the truthfulness of a complete negative is a cast iron bitch. You asked for examples of how they were used - THAT was a legit question. We could provide very few answers (e.g. Creatures).

Is there "proof" that they are not a good choice? Find me proof that a combination of pickle juice and chocolate syrup doesn't make a good topping on anchovy pizza. About the only "proof" you could get is anecdotal evidence that people simply don't like it. They all have their reasons and no amount of prodding by you will get past the notion that "it just doesn't work for me". Are they all wrong? They must be because no one provided "proof".

I'm not discouraging you from your research. I'm answering your questions. As I stated originally... be careful what you wish for.
0

Share this post


Link to post
Share on other sites
To be honest I don't understand why ANN's are used at all. For the most part they are used as a glorified regression tool. I have spent a bit of time researching a few of the different networks and the only one that struck me as interesting was the Hopfield (If I remember correctly) network. I would be willing to bet 1000 bucks that if we just changed the name from ANN to something less suggestive, much less people would be interested.

I would love for someone to give me reasons why my opinion should change though.
0

Share this post


Link to post
Share on other sites
Thanks for your opinion, you obviously dont agree that NN play a part in the games industry (or ever will) once I have done my research I will be sure to let you know my conclusion.

At least if I find a niche in the market, I will be a millionaire ha ha ha and if I dont, at least I wont waste my career trying to use a square wheel :-)

Thanks again for your contributions


0

Share this post


Link to post
Share on other sites
Quote:
Original post by sion5
Are NN's good or bad in games?


Certainly depends on the game. However, the common answer is "almost universally bad".

Why:

1) single-player "campaign" type games are very scripted environments. A very small subset of the rules design wants for the game I'm working on now:
- when the player destroys this Item, the AI will counter-attack.
- When the AI squad gets to 20% of its original units -> retreat.
- AI should only throw grenades when no other AI is throwing a grenade and has not for at least 10 seconds.
- Only 1-3 AI should ever rush the player at any given time
- AI should only spend 30% of their time hidden behind cover; further no single AI should remain hidden in cover for more than 3 seconds
- When the player's units get to a Point of Interest they should stop and play a canned set of animations at a specific point to develop the plot.

2) In a campaign environment design (and the player) want predictability. The point of game AI is not to be "smart" it is to be "fun". Fun means finding a way to exploit the AI so you can win every time. While encounters do not need to play out exactly the same they should play similarly enough that the player can learn how to beat it each time he plays the encounter and dies.

ANNs are really bad at both of those and are thus completely unsuitable for typical single-player type campaigns (aka Halo, Call of Duty 4, etc)

Things they are potentially useful for: fun interactive pets (Black & White's god avatar pet-thing), multi-player bots (there was a very sucessful Quake-bot driven by an ANN), some new genre of game.

Always remember the point of AI in games is to support that game's specific mechanics. If an ANN is the best choice then it's certainly ok to use it. It is, however, just a tool and as such is not going to be useful in many situations.

I think the project will be very interesting to you, but it's kind of representative of the chasm between academia and the professional game development world. In the former, the point of AI is to explore boundaries of flexible decision making. In the latter, the point of AI is to do exactly what the designer wants it to do.

-me
0

Share this post


Link to post
Share on other sites
I don't have a reference to cite, it may have been an earlier post here, but one of the things that NNs are not good at is dealing with sequences of events. So, you probably could find some game "type" where an NN would work really well. Perhaps someone here will suggest one.

I know that NNs are fairly good at learning to discern various distorted characters in a CAPTCHA image, for instance, but that is hardly a good "game", is it?
0

Share this post


Link to post
Share on other sites
Quote:
Original post by ID Merlin
I don't have a reference to cite, it may have been an earlier post here, but one of the things that NNs are not good at is dealing with sequences of events.

That is correct. One way around it would be to have some of your inputs map to historical data at discrete time periods (e.g. 1 second ago, 5 seconds ago, 15 seconds ago) or to discrete points on a historical event list (e.g. CurrentEvent - 1, CurrentEvent - 2, etc.)

Quote:
I know that NNs are fairly good at learning to discern various distorted characters in a CAPTCHA image, for instance, but that is hardly a good "game", is it?

The reason for this is that NNs are suited to pattern matching. For example, an OCR system will take pixelized data and use each pixel as an input. Depending on which pixels are on and which are off, it can make a reasonable assumption of which letter or number it is. It is saying, in essence, "this kinda looks like this known pattern which I have mapped to output 'A'".

For game purposes, you are not using pixels but data inputs from the world around you. Health, damage per second, number of allies, number of enemies, proximity of powerups, etc. That's great and all, but I can build the same decision model with weighted sums and/or decision trees... and in a more custom-tailored way.

But the NN can learn on the fly, you say? So can reinforcement learning or dynamic decision trees. And they can do it in a more controlled way.

*shrug*
0

Share this post


Link to post
Share on other sites
Colin McRae Rally 2 used a neural network to drive the AI cars.

I also belive that Battlecruiser 3000AD used one as well.

And the afore mentioned Black & White.

Aside from those 3 games, I'm not really aware of any others, not to say there aren't any.

CMR2 is in my mind, the perfect kind of use for a neural net though.

0

Share this post


Link to post
Share on other sites
Ok, some intresting arguements there which I will be sure to pick up on in my writting.

Forgetting now that im actually using NN's to automate a game agent(which evidently is not the best way to solve that particular problem). Is it fair to say that NN's are good at what they do, but its difficult to find an application for them?

Andre'LaMothe of Xtreme Games worte an article about NN's and one thing that struck me was this part:

"The key to unlocking any technology is for a person or persons to create a Killer App for it. We all know how DOOM works by now, i.e. by using BSP trees. However, John Carmack didn't invent them, he read about them in a paper written in the 1960's. This paper described BSP technology. John took the next step an realized what BSP trees could be used for and DOOM was born."

So it took BSP 33years to go from theory to practice! Dont get me wrong im not expecting to be creating the next Doom using NN's but maybe the arguement could be that their not being applied correctly?

0

Share this post


Link to post
Share on other sites
Quote:
Original post by sion5
Forgetting now that im actually using NN's to automate a game agent(which evidently is not the best way to solve that particular problem). Is it fair to say that NN's are good at what they do, but its difficult to find an application for them?

No, it's the other way around. It's easy to find applications for NN's - they are, in a sense, the ultimate heuristic - but they are so general that they don't perform specific operations well.

The biggest drawback is one I haven't seen mentioned here, and that is that backprop NN's tend to get stuck in local minima. This is a critical weakness. It can be alleviated with various tricks such simulated annealing or other hill-climbing algorithms, but the more tricks you pull the less efficient the learning procedure becomes. The ultimate expression of this is using GA's, which you asked about. Using GA as a learning algorithm avoids the local minima problem almost completely, but you pay for it by throwing away all structural knowledge you have of the problem which manifests itself as extremely poor learning rates.

NN's may become useful if we learn how to compensate for their weaknesses without sacrificing their strengths. It has been demonstrated that NN's and other statistical methods benefit greatly from appropriate preprocessing mechanisms, but they still do not approach hand-coded methods for stability, correctness or efficiency. In the end, most statistical methods disregard the actual structure of the problem they are trying to solve, which is why they fail. Classical AI techniques are typically better suited; a combination of the two may be best (but is difficult to achieve; I'm working on that problem myself).

Then there are statistical methods that allow for structured learning as a compromize. I've had some early success (didn't pursue it further) with Bayesian networks implemented with a learning algorithm. The price in runtime performance was heavy (superexponential in regards to the number of nodes, IIRC), but the rate of learning as a function of observed data was relatively impressive.
Quote:
So it took BSP 33years to go from theory to practice! Dont get me wrong im not expecting to be creating the next Doom using NN's but maybe the arguement could be that their not being applied correctly?

Could be. It is evident that NN's are very powerful - just look at ourselves. As I mentioned, NN's benefit greatly from proper preprocessing. Preprocessing can be (but usually isn't) done other NN's. This suggests that a tightly controlled hierarchical structure of NN's can achieve better results than we've seen so far. It makes sense from a theoretical perspective too, though I won't go into that here. But to actually design such a hierarchy properly is not possible at this point in time. Due to the nature of NN's there are no structural design methods to apply, and the solution space is much too large for random exploration to see any real success. I doubt much progress will be made withing the next two decades in this particular area, unless a breakthrough of Einsteinian proportions is made.
0

Share this post


Link to post
Share on other sites
Thanks Hnefi, that was a very informative post. I dont understand the later part of this comment:

Using GA as a learning algorithm avoids the local minima problem almost completely, but you pay for it by throwing away all structural knowledge you have of the problem which manifests itself as extremely poor learning rates.

I undertand the avoiding of the local minima problem but what do you mean by saying you do so at the cost of throwing away all structural knowledge you have of the problem which manifests itself as extremely poor learning rates??
0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0