Sign in to follow this  
astrogrrl

First person shooter AI

Recommended Posts

astrogrrl    122
Hi all =) I’m doing some research in first person shooter AI and am interested in people’s experiences and opinions on the topic. Mainly I’m interested in knowing where you think the main areas of improvements in bot AI are.

Share this post


Link to post
Share on other sites
dawidjoubert    161
Realistic Response to Sound/Objects!

Example dropping a can next to a bot needs it to look at the can, walk 2 it, inspect it, then it will suspect an enemy and look around desperately.. If the object that was dropped is anything of relevance like a clip then the bot would warn the other bots.

Sound... Mmm Consider throwing a rock, the bot doesnt see it but hears it, so he would either

A : If relaxed inspect like above
B : If Tense sound the alarm.

Thats basically it for above normal things, far cry has nice AI because the bots sometimes get frantic and just shoot where ever they think you are... Some times they are right :-)

--- EDIT ---
Oh and if the bots are Tense (alarm has sounded) they wont walk alone or stand in a 2 open spot, or enter caves... ect :-)

And if the bots have the player trapped then a realistic action would be to simple wait for him to come out :-)

Share this post


Link to post
Share on other sites
meeshoo    512
well, i think some improvments may be in the pathfinding area, because a lot of bots get stucked here and there these days. If you want to create realistic bots, please check out the FEAR SDK, it is really great. There is a book on that also, written by the author: AI Game Development by Alex J. Champandard. This book(I've got it myself) it's not for the beginner in AI, but it's really great.

Share this post


Link to post
Share on other sites
xEricx    572
Definitely what I would have liked to add in our game was more interaction between the agents and the environment. Sure our bots reacted to sounds / rocks thrown in their perception radius, but that's just icing over the top of the AI.

The problem with environment interaction often means more animations, which have to be streamed and handled correctly to fit in RAM, and requires animators and integrator to get the final result...

Share this post


Link to post
Share on other sites
dawidjoubert    161
Eric, when you say your game do you refer to Far Cry? (IE were in anyway involved)

I thought Far Cry's ai was good enough, and just when i figured out the trick, i have to start fighting mutation thingies!!!

How ever the most coolest thing about Far Cry was its introduction of a Basic yet in realtime games term Elite Physics engine!

----------
To top it up, Battle Field 2's physics engine is really lacking... Have u ever tried driving one Humvee into another one at 30-50mph and then when u hit the object in rest nothing happens you both just stop!!!

INSTEAD of ur humvee basically catapulting into the air!!.. its really sad cause that effect would have rockeD!

Share this post


Link to post
Share on other sites
Steadtler    220
I always pondered about the power of scenarios in bot games.

An AI scenario is something like

Player enter Room A
Player flip the switch
Room C fills with water
Player leaves room A

Using that scenario, a bot that sees the player exiting Room A can deduce that the player may have flipped the switch and that room C may be filled with water.

Its a very simple example, of course.

Share this post


Link to post
Share on other sites
xEricx    572
I did not work on the original Far Cry title, which was made in Germany. I worked on the greatly modified version called Far Cry Instincts, for Xbox :)

Steadtler, I believe most of this kind of stuff can and should (most of the time) be faked. Designers love to have deterministic situations and to be able to know what the AI knows at what point of the game... leaving the AI to deduce things is indeed, interesting, but could be hell in debugging, for almost no gain (since it can be faked and controlled).

Astrogrrl: What you're looking for is improvements on the coding / way to represent data / decisions or the in game results?

Cheers

Eric

Share this post


Link to post
Share on other sites
Steadtler    220
Quote:
Original post by xEricx
Steadtler, I believe most of this kind of stuff can and should (most of the time) be faked. Designers love to have deterministic situations and to be able to know what the AI knows at what point of the game... leaving the AI to deduce things is indeed, interesting, but could be hell in debugging, for almost no gain (since it can be faked and controlled).
Eric


I agree, but players hate deterministic situations and cheating AI. Scenarios encode preconceptions about how things are done. It would allow a bot to seem more believable, AND be more intelligent without cheating. Also, deterministic AI makes it more predictable by the player.

Share this post


Link to post
Share on other sites
astrogrrl    122
Eric: I am still in the early stages of research so trying to figure out an area I can improve on. Ultimately I would like to improve either the overall behaviour or a certain aspect of the bots (so in game results to answer your question), whether this is through a different way of representing the data or decision making process I'm not sure yet. The scenarios Im tossing around are deathmatch or team based ones.


Meeshoo: Thanks for the links, Ill check out the FEAR sdk. I also have to decide on an engine to use, well when i get to the development stage. I have also got Champandards book and got a lot of my inspiration from him ;)


Steadtler: You have brought up a very interesting point, deterministic == predictable, but developers need deterministic rules for debugging which is completely understandable. I just had an idea, what if the bot built up scenarios dynamically during gameplay? Then they could refer to situations they are in which are similar to ones they have experienced before. So essentially they learn through what they see and experience. Though this may break the deterministic rule, potentially some interesting gameplay could come out of it.

Share this post


Link to post
Share on other sites
Timkin    864
We need three things to dramatically improve the quality of FPS AI.

1) We need bots that interact with players on a psychological level. That is, rather than being passive reactants to the players interactions with the game, they should actively exploit the players mood to affect gameplay. Mood can be estimated through actions. If the player is racing all over the place, shooting at anything they see, they're probably hyped and excited (the adrenaline is pumping). If they're hiding behind a crate, they may be relaxed (if sniping) or tense, if setting an ambush. Using this sort of basic analysis and changing bot tactics and strategy would dramatically alter the experience one has in an FPS setting.


2) We need to retrain designers so that they realise that 'deterministic' and 'unpredictable' are not mutually exclusive. That is, something can be completely deterministic and yet very hard to predict, simply because of its complexity (the heart of chaos theory). Giving designers the tools to explore this complexity will open their eyes to the possibilities. Giving them the tools to funnel the evolution of the game state through disjoint narrow windows will enable them to generate storylines that meet certain design expectations, yet have sufficient room between the windows to expand into into temporary free-form gameplay.

Finally,

3) BoTs should think like players (and not like BoTs). By that, I don't mean they should be as intelligent, but they should play the game like a player does. They should vary their tactics depending on what their opponents are doing. They should suffer from human frailties (uncertainty, arrogance, fear, loathing, a desire for retribution, etc). Imagine being camped by a bot simply because you pissed him off by camping him!? Sound familiar?

Astrogrrl, if you're interested in making FPS bots a little more human, you might be interested in Emma Norling's work (and that of the Intelligent Agents lab at Melbourne Uni). They do a lot of work on AI for military training simulation environments. Emma did her PhD in this area, but due to the confidential nature of the sim environment, she implemented a lot of her ideas in Quake first so she could publish. Her work might give you some inspiration.

Cheers,

Timkin

Share this post


Link to post
Share on other sites
xEricx    572
astrogrrl, ok, sorry then I have no experience with "bots". AIs that populate a world to help out the scenario are a totally different thing in my mind than bots. They both have their complexities and I'm not really experienced with bots.

This link might be interesting to you, it's William van der Sterren's web page, it contains a few interesting links.

Hope this helps

Eric

Share this post


Link to post
Share on other sites
Steadtler    220
Quote:
Original post by astrogrrl
I just had an idea, what if the bot built up scenarios dynamically during gameplay? Then they could refer to situations they are in which are similar to ones they have experienced before. So essentially they learn through what they see and experience. Though this may break the deterministic rule, potentially some interesting gameplay could come out of it.


For linear scenarios, this is rather easy. You can search common sequences in observed player actions. For exemple, the bot could observe that the player always grab the sniper rifle and head west after respawning at location B. So if the bot hear a sniper rifle shot, it can look west of location B for the player. Or if it sees the player west soon after killing him, it can assume the player has a sniper rifle. We all have those habits when playing action games, all have our favorites way around maps... It would be interesting for a bot to observe this and vary its strategy accordingly.

But scenarios become really interesting when non-linear, and thats when they are complex to observe.

Share this post


Link to post
Share on other sites
Nice Coder    366
One thing that would always be interesting to me is interbot communication.

Ie. your running away from one bot (you need to achieve your objective quickly, or your running out of hp/ammo), and that bot tells another bot where you are.

That bot happens to be around the next corner, so it stops, backs up onto the wall, switches to shotgun, and shoots you in the back the second you come around.

Also, getting bots to call in for backup would be pretty nice. Just after you machinegunned a few bots, a few more come from behind and kill you.

Getting bots to divide fire would also be a nifty little attribute. Instead of everyone firing at whoever is closer, they all find there targets and shoot at them.

From,
NIce coder

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
Quote:
Original post by Timkin
We need three things to dramatically improve the quality of FPS AI.

1) We need bots that interact with players on a psychological level.


Hook up a neural probe to the player?? ;-)

Quote:

2) We need to retrain designers so that they realise that 'deterministic' and 'unpredictable' are not mutually exclusive.


The main problem with non-deteministic, unpredictable agents is that it becomes very hard to test them. It would be very embarssing to the company if they shipped a project and there bots did something quite bizarre. But I agree,
it is the future of game AI!

Share this post


Link to post
Share on other sites
Timkin    864
Quote:
Original post by Anonymous Poster
The main problem with non-deteministic, unpredictable agents is that it becomes very hard to test them.


You didnt read what I wrote correctly. I said deterministic and unpredictable are not mutually exclusive. I did not say that agents should be non-deterministic.

You can have completely deterministic bots that are difficult to predict (if you don't have all of the information regarding their state and/or their agent function). In development and testing, where you know their state and function exactly (because you can look inside the bot with your design tools), you know exactly how it is going to behave given any specific stimulus. To the player though, who cannot access the agent function or assess the game state perfectly, they can only guess the agents behaviour. The degree to which this guess is accurate will depend on how well they have observed previous behaviour, how well they have assessed the game state and the nature of the agent function.

There are very specific mathematical measures for edge-of-chaos behaviour that can be used in the design process, so there is no need for any company to ship a title that they don't know exactly how it will behave. It just requires education and a willingness to extend what is currently the norm.

Timkin

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
Quote:
Original post by Timkin
Quote:
Original post by Anonymous Poster
The main problem with non-deteministic, unpredictable agents is that it becomes very hard to test them.


You didnt read what I wrote correctly. I said deterministic and unpredictable are not mutually exclusive. I did not say that agents should be non-deterministic.

Timkin


That sounds like cool stuff Mr. Timkin, maybe you could write up a small article on how this is done? It's hard to understand how it isn't mutually exclusive.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
Quote:
Original post by Timkin
We need three things to dramatically improve the quality of FPS AI.

1) We need bots that interact with players on a psychological level. That is, rather than being passive reactants to the players interactions with the game, they should actively exploit the players mood to affect gameplay. Mood can be estimated through actions. If the player is racing all over the place, shooting at anything they see, they're probably hyped and excited (the adrenaline is pumping). If they're hiding behind a crate, they may be relaxed (if sniping) or tense, if setting an ambush. Using this sort of basic analysis and changing bot tactics and strategy would dramatically alter the experience one has in an FPS setting.


2) We need to retrain designers so that they realise that 'deterministic' and 'unpredictable' are not mutually exclusive. That is, something can be completely deterministic and yet very hard to predict, simply because of its complexity (the heart of chaos theory). Giving designers the tools to explore this complexity will open their eyes to the possibilities. Giving them the tools to funnel the evolution of the game state through disjoint narrow windows will enable them to generate storylines that meet certain design expectations, yet have sufficient room between the windows to expand into into temporary free-form gameplay.

Finally,

3) BoTs should think like players (and not like BoTs). By that, I don't mean they should be as intelligent, but they should play the game like a player does. They should vary their tactics depending on what their opponents are doing. They should suffer from human frailties (uncertainty, arrogance, fear, loathing, a desire for retribution, etc). Imagine being camped by a bot simply because you pissed him off by camping him!? Sound familiar?

Astrogrrl, if you're interested in making FPS bots a little more human, you might be interested in Emma Norling's work (and that of the Intelligent Agents lab at Melbourne Uni). They do a lot of work on AI for military training simulation environments. Emma did her PhD in this area, but due to the confidential nature of the sim environment, she implemented a lot of her ideas in Quake first so she could publish. Her work might give you some inspiration.

Cheers,

Timkin


Well c'mon TImkins, you make it sound so easy, let's see you put your money where your mouth is! Let's see it!

Share this post


Link to post
Share on other sites
Timkin    864
Quote:
Original post by Anonymous Poster
Well c'mon TImkins, you make it sound so easy, let's see you put your money where your mouth is! Let's see it!


Such antagonism and rudeness is not called for. If you want more information you simply had to ask for it politely, rather than 'challenging' me publicly. You seem to think that I'm just sprouting bullshit to be heard... you're obviously new around here... but I'll put your ignorance aside and answer your question. (btw, double posting is not permitted... yes, I can see your IP address).

The answer is easy, it just requires that you understand something of dynamic systems, which is something that most game AI programmers and designers lack (since most are just average programmers with little or no explicit education in AI or mathematics). Thankfully many of them do display a willingness to learn more about their craft. If they're reading this, then they're welcome to this information (since it's fairly common knowledge in formal AI/control circles).

Any state machine, during its operation, traces out a defineable trajectory in both state space and in phase space. Trajectories in phase space define a vector field and this can be analysed for its divergence considering the magnitude and sign of the Lyapunov exponents of the system. Positive exponents represent divergent energy in the phase space trajectories. Negative exponents represent convergence. One of the requirements for chaos is that the largest positive exponent have greater magnitude than the most negative component. If they are (nearly) equal then you get 'edge of chaos' behaviour.

Irrespective of whether or not the state trajectory is chaotic, one can define its behaviour for all time by identifying the phase space attractor that governs the state space trajectories. Designing and tuning state machines with a view to their phase space properties gives designers far more powerful tools to provide a deeper understanding of the dynamics in state space.

It's fairly trivial to write down deterministic (and simple) mathematical expressions that produce bounded, yet chaotic behaviour. Knowing the state of the system at any instant the succeeding states can be predicted exactly, without any resort to approximation. However, if the system is chaotic (or even just nearly chaotic') then small errors in the knowledge of the state will show as a divergent predicted state trajectory from the true one. Designers can access exact state information to predict the behaviour of their bots exactly. Game players cannot do this. Depending on the rate of divergence in the local region of the state space they are currently in, the temporal horizon over which they can accurately predict the state evolution will be limited.


Timkin

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
Quote:
Original post by Timkin
Quote:
Original post by Anonymous Poster
Well c'mon TImkins, you make it sound so easy, let's see you put your money where your mouth is! Let's see it!


Such antagonism and rudeness is not called for. If you want more information you simply had to ask for it politely, rather than 'challenging' me publicly. You seem to think that I'm just sprouting bullshit to be heard... you're obviously new around here... but I'll put your ignorance aside and answer your question. (btw, double posting is not permitted... yes, I can see your IP address).



Well, gee I'm real sorry. Please forgive me.


Quote:

The answer is easy, it just requires that you understand something of dynamic systems, which is something that most game AI programmers and designers lack (since most are just average programmers with little or no explicit education in AI or mathematics).


Gee, it sucks being a lame, average programmer.

Quote:

Any state machine, during its operation, traces out a defineable trajectory in both state space and in phase space. Trajectories in phase space define a vector field and this can be analysed for its divergence considering the magnitude and sign of the Lyapunov exponents of the system. Positive exponents represent divergent energy in the phase space trajectories. Negative exponents represent convergence. One of the requirements for chaos is that the largest positive exponent have greater magnitude than the most negative component. If they are (nearly) equal then you get 'edge of chaos' behaviour.


Uh-huh. That sounds *real* easy. Like I said, let's see it *work* and not just a bunch of bullshit about how easy it actually is...


Share this post


Link to post
Share on other sites
jbadams    25676
Quote:
Original post by Anonymous Poster
[...]

Cut the attitude. Timkin has remained polite in spite of your behavior, and I'm sure is perfectly happy to provide more information to those who're genuinely interested. He may not have the time and/or inclination to provide a working example implementation, but let me assure you that he knows the subject area and wouldn't be providing this information without the knowledge to back it up.

Discussing the matter at hand is fine, and even disagreeing with something that's been said is still fine, but be polite, and if you're going to disagree, try to back up your assertions with something other than "it's bullshit".

The "bullshit" by the way seems to make sense to me in spite of having no particular affinity for AI - I suspect you may not have the background knowledge required to make sense of it - perhaps you could do some research into the area so that you'll be able to understand, or politely ask for more information or a reference to a resource you could look at rather than harassing Timkin.

Now, be polite from now on, got it?

Share this post


Link to post
Share on other sites
xEricx    572
Timkin, I tried a quick google search on "Lyapunov exponents" which I thought wouldn't give me that much hits... I was wrong. hehe.

Do you have any "introduction" books / papers to recommand?

Thanks a bunch.

Eric

Share this post


Link to post
Share on other sites
Timkin    864
Quote:
Original post by xEricx
Do you have any "introduction" books / papers to recommand?


That really depends on what approach you want to take. Most introductory texts on dynamical systems should touch on issues of stability (and hence on Lyapunov exponents). Books on Chaos theory that provide mathematical analysis will do the same. You'll also find a wealth of information in control theory literature on stability analysis.

As to good books in each of these fields, that's subjective and depends on your background. I tend to use textbooks and journal papers, which aren't necessarily good introductory material. My recommendation for an absolutely introductory view of this material would be from the perspective of chaos theory. For an introduction to some of the key concepts you could try Abraham, Gardini & Mira: "Chaos in discrete dynamical systems". It has lots of pictures to explain concepts and doesn't use much mathematics beyond very basic stuff.

A nice, in-depth an easy to read look at the mathematics for describing and analysing certain aspects of dynamical systems (especially chaotic systems) is

Beltrami: "Mathematics for Dynamical Modelling"

A more intermediate level book would be something along the lines of

Solari, Natiello & Mindlin: "Nonlinear Dynamics",

while an advanced book bringing together works from many leading researchers is

"Control & Chaos", edited by Kevin Judd, Alistair Mees, et al.


Once you have an understanding of the analysis of dynamical systems, it then becomes a question of how do you design dynamical systems such as state machines and fuzzy state machines so that they display certain dynamic characteristics. This is where the control theory literature comes in. Thom Dean and Michael Wellman's book "Planning and Control" is a really good book, although not an introductory one. "Essentials of Fuzzy Modelling and Control" (Yager and Filev) is fairly easy to read, although it doesn't cover analysis of the system dynamics. I have yet to find a good book that encompasses all of the relevant concepts.

Anyway, here's a really simple example of how you might use knowledge of nonlinear dynamics in a bot design problem. Let's say you want a bot to walk in a maze of corridors (such as a FPS game level). At each intersection, it has the choice of going left, right or straight (or a subset of these choices depending on the topology of the corridor network). We don't want the bots path to be particularly predictable... or we want to be able to vary its predictability as a function of a design parameter. So, we choose, for example, the Logistic Map: x(i) = r*x(i-1)(1-x(i-1)), where x(i-1) is the direction chosen at the previous intersection and x(i) the direction to go at the current intersection. Now, for the logistic map, x is a positive real number on [0,1] iff r is a positive real number on (0,4]. Hence, if we want to map x onto a direction decision variables, we might simply choose x<=0.33 => left, 0.33<x<=0.66 => straight and 0.66<x<=1 => right. For a more complicated decision variable, or a multi-dimensional decision problem, we could use a Fuzzy mapping from x to the discrete state/action variables. If you consider the logistic map for different values of r and different initial conditions, you'll find that you start to get period doubling for r>3. Taking a value close to 4 gives very unpredictable behaviour (if you don't know that the logistic map is driving the decision process). Obviously though, the level designer knows this, knows the value of r and the last decision state, so they can exactly predict all subsequent states (and tune the behaviour through the parameter 'r' to achieve the degree of predictability they want. They can even go so far as to ensure that the bot passes through a given sequence of intersections, or reaches a particular intersection at a particular iteration). Adjusting the parameter 'r' based on what the player does, or what other bots do, would also add a level of depth to the behaviour that would certainly enrich the gameplay.

Of course, there are other ways to use dynamical systems analysis to improve the design process. For example, understanding how outside forces (perturbations) will influence the evolution of a bots state machine, or designing unpredictable dynamics into bots but keeping their behaviour constrained to a finite volume of state space (by appropriate design according to the phase space attractor).

These principles have been used for a long time in the design of control systems and the analysis of dynamical systems (which all controlled plants are examples of). It would be nice to see some of this very widespread knowledge actually adopted into games.

Cheers,

Timkin

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this