Archived

This topic is now archived and is closed to further replies.

alexjc

Role of Learning or Adaptive AI

Recommended Posts

alexjc    457
Branched from a comment from fup in the AI Programmer's dinner thread. > I'd like to know how developers feel about the role of > 'learning' or adaptive AI in future projects. I was very interested an enthusiastic about this myself, so made a point of bringing it up whenever possible to professional AI coders (before or after the ritual exchange of business cards). Frankly, I was appauled and shocked. Not only do most game AI programmers not really care about learning techniques, if they could do without AI to create NPC behaviors, they probably would... (we won't get a balanced discussion here as we're all in the same camp, but it's true) In the industry, the whole mentality of creating a game is about designer control, and that makes ANY adaptive techniques extremely unappealing. With better AI, designers become less useful, testing is awkward... etc. Some companies are making the effort, but these require fundamentally different design techniques, so I can't see them catching on in the near future. Bad news for us AI enthusiast I'm affraid. I percieve a general "our solution is good enough" attitude, "so why change"? (I got the same feeling after my lecture on super-duperly advanced amazingly awesome navigation system :] "But A* works ok!") Removed off topic comment. [edited by - alexjc on March 18, 2003 7:23:11 AM]

Share this post


Link to post
Share on other sites
Timkin    864
Resistance to change is not restricted to the games industry, although it IS particularly prevalent throughout that industry. Companies require a ''winning formula'', tried and tested and ''guaranteed'' to churn out a product in N days of production.

Learning systems and adaptive AIs are unknown quantities in the industry, in that very few people have any real understanding of them, let alone having experience with them.

There are, however, companies that are investing time and money in developing adaptive game systems. Don''t expect to see them on the shelves any time soon, as they are more long term projects.

For now though, independent developers will be the ones to set the pace in the advancement of game AI, because they (usually) have the time to put into the project and are not restricted by budget and production constraints. This means that we, as a community, should be working hard at improving our game AI. If WE can prove that a particular technique is stable, reliable and moderately easy to implement into an existing code base, then the industry will sit up and listen.

So, keep at it and let the good AI roll on!

Cheers,

Timkin

Share this post


Link to post
Share on other sites
fup    463
I''m not surprised given the round table discussions from previous years. I was just interested to see if there had been any change in the last year.

I agree with Timkin. I feel we will see adaptive AI become more prevalent as time goes on. Once learning technologies start to get some exposure and developers become more comfortable with them (and as AI programmers become better educated) I have no doubt they will become widely used. Primarily because they provide an ''edge''. This is important in today’s cut throat industry.


PS. Alex, if you keep slagging off NNs I''m gonna come up North and give you a slap





ai-junkie.com

Share this post


Link to post
Share on other sites
alexjc    457
> Once learning technologies start to get some exposure and
> developers become more comfortable with them

Bah! There has been focus; some best selling games... that still hasn't made any difference. What is needed is learning techniques that can be controlled, overriden and managed by designers, which almost beats the point of spending time on adaptive AI (certaintly at this point + couple years).

Combine the two (top-down / bottom-up), and you're sorted. Lots of research going into this. We discussed this on my part of the table at the AI Prog. Dinner -- one of the good discussions


Removed off topic comment.

[edited by - alexjc on March 18, 2003 7:24:01 AM]

Share this post


Link to post
Share on other sites
fup    463
"Combine the two (top-down / bottom-up), and you're sorted.

Well yes, that much is obvious.

"and NN are overrated"

Certainly not by the games industry.




ai-junkie.com

[edited by - fup on March 18, 2003 6:25:52 AM]

Share this post


Link to post
Share on other sites
HG    122
I think adaptive techniques will only find a place where a designer cannot feasibly do the job - agents on the Semantic Web, for example. That''s how most designers work.
The real problem is that in games, it''s frequently much easier to do things more rigidly; you avoid the situation where the adaptive AI works around the rules you''ve imposed, and you fight the problem, not the learning AI.

Put these two together and you can probably infer that Doom III will use waypoints and state machines, and SimCity 6 will use more agent-like techniques for its sims.

Adaptive techniques might also find a role in a less-important area - maybe NPCs learning reactions - but I wouldn''t expect adaptive end bosses

Share this post


Link to post
Share on other sites
Geta    136
One area where I personally have tried to employ adaptive techniques is in the strategic decision-making of the programmed opponent or artificial computer player (AI Player) of RTS/Strategy games. Both in Enemy Nations (a RTS) and again in my series of shareware, turn-based wargames (the Battles series), I tried to have the AI Player perceive (using influence mapping and pattern matching techniques) the human opponent''s tendancies and then have the AI Player change its goals to attempt to counter what it thought the human was doing. The results I observed were less than acceptable, due (IMO) to problems with the AI Player''s perception of the human''s tendancies.

All that being said, I think that using adaptive techniques in the AI Player is of interest to RTS and Wargame AI Programmers. However, I think that bot control in FPS games is where we will see the most progress at using adaptive techniques in commercial games, simply because there are more people experimenting and tinkering with bot AI than other forms of computer game AI.

Eric

Share this post


Link to post
Share on other sites
alexjc    457
The major problem is establishing a sandbox -- by design or implementation. This needs to be done in order to make sure that nothing goes wrong (as decided by the designer).

For different applications, this is more or less difficult to do (implicit & explicit AI techniques), and it may in fact take as long to develop the sandbox as the adaptive technique itself!! Hence the apprehensions...

So, back to what you were saying Eric, it seems RTS would need less work to build/design that sandbox (a few consistency checks before lanching an attack). So it seems easier to create consistent behaviors -- than for an FPS bot for example.



Artificial Intelligence Depot - Maybe it''s not all about graphics...

Share this post


Link to post
Share on other sites
IADaveMark    3741
We are going to be experimenting with something simple on our game. We are going to be formulatically tweaking some of the response curves that we are using to decide behavior. So, as the game conditions change - in part because of the player''s strategy, those response curves will be modified slowly so that the competing AI''s decision structure will change as well.

Dave Mark - President and Lead Designer
Intrinsic Algorithm -
"Reducing the world to mathematical equations!"

Share this post


Link to post
Share on other sites
RPGeezus    216
It's been my experience (in the non-gaming industry) that most people reject AI methods outright. They feel that it is to wishy-washy, and at home only in academia.

I think part of the problem is that for most people their introduction to AI comes in the form of systems such as COG, Deep Blue, or fancified ELIZA's. You rarely hear about actual successful implementation of an AI method that would solve a problem most programmers could relate to.

As far as the gaming industry goes, I asked a friend of mine who works for one of the game console companines. He told me that 'AI' is usually a very low priority on the list of things to do. Every project he has been involved in has used state tables mainley because they are easy to implement and easy to customize.

Another part of the problem, I would suspect, is the gap between academic research and industry publications. Dr. Dobbs, for example, tends to be a little less formal than your average research paper. The language is usually aimed at a wider audience and written in a way that facilitates quick digestion.

There is also a lot of bad-press associated with the term AI. There have been too many promises of HAL-like computers over the past twenty-years. I think if AI techniques are ever to really catch on, they will not be refered to as AI. I think a recent article on Salon.com talks about this.

Cheers,
Will


[edited by - RPGeezus on March 18, 2003 10:47:49 AM]

Share this post


Link to post
Share on other sites
IADaveMark    3741
First off, that article from Salon.com - not unlike most of their other offerings, was dog shit. Sure, that Loebner dude is a whacked out loser, but there was not a lot of objective content in there. The writer also kinda sucked ass.

As for academia, they have their place, but they are currently trying to solve different problems than is the gaming industry. The gaming industry IS paying a lot more attention to AI these days... ask Ferretman for the research numbers.

The twist is, however, that we are still searching for the APPEARANCE of intelligent behavior rather than the perfection of intelligent behavior. The time frame for most game sessions is not great enough - nor the depth of most gameplay vast enough, to explore beyond a simple, narrowly focused, facade of AI. Not too many people spend time in Madden 2003 wondering what sort of psychological impact it would have if your star defensive tackle were to use "yo momma" taunts at the line of scrimmage. Does it happen in real life? Sure. Does it have an impact on personal performance and therefore on the outcome of the game? Possibly. Is it necessary to model that in Madden 2004? Not likely.

One of the drawbacks on using "the perception of intelligent behavior" as a yardstick is that "perception" is a nebulous, highly subjective concept. It does not lend itself to quantitative ananlysis and categorization. If you were to sit a 4 year old, a 65 year old, and a 25 year old non-computer user down in front of the computer, 90% of the major titles on the market would pass the "Game Turing Test". They would be fascinated by what they see and could be easily and thoroughly convinced that there was a human on the other side. If you fill the room with AI assholes like us, we would pick 90% of those same titles apart. Obviously, there is a large hump in the bell curve between the two extremes. Which segment of the game playing population are we trying to convince?

My contention is that there is a significant chicken/egg syndrome involved. As I mentioned, most game designs do not have the scope of play necessary to require or utilize adaptive/learning techniques. Is that because they don''t believe that there is significant technology to do such things? Perhaps the technology hasn''t been pushed because there aren''t suitable games to use it? Which should come first?

"Black & White" was notable in this sense. It gave us a large scale sandbox in which to experiment with their very "out front" learning AI system. They proved their point admirably that the creature COULD learn from it''s environment - part of which involved an eerily detatched hand smacking it around on occasion. While B&W showed that it could be done, the trick is to now take what was learned in that process and apply it in less prominent ways into the architecture and design of future games. Will Lionhead be the only bannerman for that foray into advanced AI or will other companies realize that there are both techniques and a market for that sort of approach? That is likely just a time-based function. Eventually, many more companies will come around to realizing the possiblities - and will slowly begin to change their game designs to incorporate such concepts.

Dave Mark - President and Lead Designer
Intrinsic Algorithm -
"Reducing the world to mathematical equations!"

Share this post


Link to post
Share on other sites
fup    463
Slightly off topic here but I''d like to mention this before I forget...

A point I''d like to make is that when discussing AI techniques for game development, the majority of people automatically assume that you are talking about the gameplay . Indeed, designing the AI for the gameplay is the fun part, but AI techniques have other uses too. Here are some examples:

B&W made innovative use of NNs in the user interface. [gesture recognition]

I know of several companies using tools utilizing Self Organizing Maps to create optimal palettes for their console/gameboy/mobile games.

I''ve also heard of companies evolving agents purely to test their physics model (because evolution almost always finds those niddly little shortcuts and bugs your play testers are never going to find)

We are starting to see GAs used in the development of character animation.

Just thought I''d bring this up as this side of AI is rarely discussed here.






ai-junkie.com

Share this post


Link to post
Share on other sites
Geta    136
quote:
Original post by alexjc
The major problem is establishing a sandbox -- by design or implementation. This needs to be done in order to make sure that nothing goes wrong (as decided by the designer).

For different applications, this is more or less difficult to do (implicit & explicit AI techniques), and it may in fact take as long to develop the sandbox as the adaptive technique itself!! Hence the apprehensions...

So, back to what you were saying Eric, it seems RTS would need less work to build/design that sandbox (a few consistency checks before lanching an attack). So it seems easier to create consistent behaviors -- than for an FPS bot for example.



I'm not sure what you mean by sandbox . Could you please elaborate?

If you are refering to what I call a testbed , then there are plenty of testbeds in the FPS game world: Half-Life and the Half-Life SDK, Unreal and UnrealScript and Quake II and QuakeC, are all available at bargin bin prices. I know of no commercial-grade RTS that would provide an equal level of access to the core code and AI manipulation to allow for experimenting and the development of adaptive AI. There is also a FreeCraft Soundforge project that is still running, I think, but I'm not aware of anything else.

If by sandbox you are referring to constraints then I would have to think about that one for a bit.

If you mean something else, then I am about to learn something new.

Eric


[edited by - Geta on March 18, 2003 4:41:55 PM]

Share this post


Link to post
Share on other sites
alexjc    457
Webopedia > S > Sandbox

A term commonly used in VM / OS development, in this case java applets.

The idea for AI is to restrict the scope of abilities to a consistent subset of safe / approved actions. Like a kid in the sandbox, hopefully nothing unexpectedly goes wrong (except eating sand and it getting stuck inside ears ). Needless to say its a lot of work, though I believe it''s applicable in a few cases... more on that soon!




Artificial Intelligence Depot - Maybe it''s not all about graphics...

Share this post


Link to post
Share on other sites
Timkin    864
quote:
Original post by InnocuousFox
As for academia, they have their place, but they are currently trying to solve different problems than is the gaming industry.


I disagree with this. Academia tends to concetrate more on the underlying theme rather than the specific application... which is why many people don''t see the point of academics! What the games industry needs is executives who are prepared to stick their noses into academic research, take a look around and assign team members to develop applications of the techniques they find there.

Geta, you would probably find some use for the techniques of ''Plan Recognition'' and ''Opponent Modelling'' if you wanted to improve your adaptive AIs for RTS games.

On the issues of having designers develop AI (for this is what we are really talking about), it is far easier to write tools for FSMs and script engines than it is for autonomous agents. However, it''s not impossible. I''ve forgotten the title and studio, but there is a group out there at the moment developing agents by building up sets of basic behaviours and having these generate non-linear behaviours in the agents (along the lines of what Brooks did for robotics).

Furthermore, one theme in adaptive AIs for the RTS genre is agents that can choose from a set of basic plans and combine them into a large variety of complex plans, based on an assessment of the opponents plan. Giving preference to shorter plans and higher payoffs makes it far easier to test the behaviours of the agent.

Ultimately though, what it will take is for faith among the games industry to embrace techniques that are provably optimal. I can offer up a proof as to why a particular technique is optimal on a given problems (but not for all problems! ), even though the agent implementing the solution will be autonomous... it will enact the optimal solution given the information available to it. This is the best that can be hoped for and in many cases, better than what is currently done. We just need the industry to accept that autonomous opponents acting in their own best interests toward designer specified goals (or adaptive goals from a specified set) are more entertaining than scripted bots guarding a powerup!

Cheers,

Timkin

Share this post


Link to post
Share on other sites
Geta    136
quote:
Original post by Timkin
Geta, you would probably find some use for the techniques of ''Plan Recognition'' and ''Opponent Modelling'' if you wanted to improve your adaptive AIs for RTS games.



I agree.

How would one go about implementing effective Plan Recognition and accurate Opponent Modelling for games like Age of Kings or Empire Earth which must operate in real time, with upwards of eight computer or human opponents, and hundreds (if not thousands) of agents to manage? And be able to process on a typical consumer system?

Any suggestions are appreciated.

Eric

Share this post


Link to post
Share on other sites
Euan    122
I think that the key to being able to use cool AI techniques in the games industry, paradoxically, is to work on games that aren''t AI-centric.

I''ve never worked on a game like Thief, but I would imagine that it would be quite difficult to do anything non-standard because the designers would need to have total control over the AI because it''s so central to the gameplay experience.

However, in something like, say, racing games, all the designers want is a giant knob labeled "Easy <-> Hard", and us AI guys do whatever we want to make it happen. As a result, I''m seriously considering implementing a learning algorithm or two (frozen when the game ships, of course) to help increase the level of "Hard" while minimizing cheating.

At the risk of sticking up for the designers, I don''t think that it''s such a bad thing that they want to exert some control. I was relieved when other people tuned the AI in our last game -- making it fun is incredibly difficult, and I sure didn''t want to have all of our hard work not appreciated because of unfun gameplay.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
If I can give my laymans 2 cents worth...

While I wouldn''t call myself an AI expert, I''ve been recently trying to get into machine learning (reinf.learning/NN/GA) a lot more
My intention is to make some rivettingly(s?) intelligent games in my spare time

Anyway, to the point...
So far my experience in this arena is indeed that there are very few articles that I can easily relate to
Most get way too philosophical or academic about it (like many have mentioned so far)
Practical articles are hard to come by, it seems
The best I''ve found as an introductory so far is www.ai-junkie.com

I agree that it is imperative that people like me do more research and attempt to implement AI in a practical, and useful (& hopefully reusable) method
The sad fact that in all the academia, it seems to me that its ''more talk that action''.. if you know what I mean
I understand that design is important, etc, but practical examples would be far more inspiring that a lot of jargon
This all makes my attempts very difficult!

Perhaps I could try help change all that by doing some more of what ai-junkie does.. offering a helping hand with a practical edge

PS: if anyone has suggestions that would make my life any easier for learning eg. RF, RF with NN & perhaps GA, pure GA, etc
it would be greatly appreciated
You can email me if you like on greg_de_haas@hotmail.com

Share this post


Link to post
Share on other sites
Predictor    198
It might help to distinguish between two very different meanings of "A.I.", as used in the game industry: "A.I." as the computer opponent and "A.I." as "intelligent" technology applied to any part of a game.

I work as a data miner and use neural networks, etc. every day. In my experience, it is sometimes more effective to use these technologies as part of the solution, rather than the whole solution. In the context of playing chess, this would mean using a neural network as an evaluation function in conjunction with a conventional game tree, as opposed to trying to use a neural network to simply output the next move. For a good example of this idea, see this paper:

http://www.iai.csic.es/users/gpa/postscript/Montpellier_202.pdf

...which describes a fuzzy logical system used to threshold images for a machine vision application. The paper is only 6 pages long and I think is fairly easy to understand (the diagram at the bottom of page 2 sums it up well, I think). The important thing to note is that the fuzzy logic rules do not generate an optimal threshold, but suggest which way (up or down) to adjust any given threshold. The system iterates, with the fuzzy rules bumping the threshold up or down until some criterion is met.

From my brief interaction with game programmers online, I suspect that their failure to effectively apply these technolgoies may often be attributed to trying to "swallow too much" by expecting the technique to solve the whole problem, as opposed to solving the part it's good at.

-Will Dwinnell, MBA
http://will.dwinnell.com




[edited by - Predictor on March 19, 2003 5:28:00 AM]

Share this post


Link to post
Share on other sites
alexjc    457
The problem is not the applicability of adaptive AI, or availability of alternative solutions, or even how long they take to understand & develop.

Game designers mostly do not want adaptive AI.



Artificial Intelligence Depot - Maybe it''s not all about graphics...

Share this post


Link to post
Share on other sites
Geta    136
quote:
Original post by alexjc
Game designers mostly do not want adaptive AI.



While this might very well be true (we should get Ferretman to do a poll on gameai.com) Game Players do want adaptive AI. For that reason, I think it is important for us as game designers to try to solve the problem (low CPU, unfamiliarity with learning algorithms, complex decision processes, etc.) that stand in the way of providing adaptive AI to the Game Players .

Eric

Share this post


Link to post
Share on other sites
alexjc    457
quote:
Original post by Geta
we should get Ferretman to do a poll on gameai.com



heh, maybe not the ideal place to get an unbiased vote from designers , but it's worth a try If you use an open question "As a designer, would you make use of adaptive AI?", then most designers will vote for adaptive AI! But, formulate it like this, and ask the programmers:

"As a programmer, is adaptive AI necessary to accomplish what designers want?"

I wouldn't be surprised if you got different results How about badgering the design section of the forums; any professionals there?


As for Game Players wanting adaptive AI, that is certainly true! But whether most GD teams have the structure, design processes and mentality to provide it is the biggest point... they arguably don't at the moment. So as AI coders, we'd be working for fun + potential long term prospects -- not short term rewards & salaries!

Adapting AI technology is a small challenge compared to what needs to be done to accomodate adaptive behaviours in professional environments (design, development process, Q&A...)


Though I'm playing devil's advocate. Most of us here would do our best to make it happen



Artificial Intelligence Depot - Maybe it's not all about graphics...

[edited by - alexjc on March 19, 2003 11:14:33 AM]

Share this post


Link to post
Share on other sites
Timkin    864
quote:
Original post by Anonymous Poster
So far my experience in this arena is indeed that there are very few articles that I can easily relate to
Most get way too philosophical or academic about it (like many have mentioned so far)



That tends to be because academic articles assume a certain level of education. While I know this sounds snobbish, it''s not. Academic publications must be kept short (due to publishing costs). The typical length is about 6-8 pages for conference articles and 10-12 for journal articles. In this limited space you don''t have room to provide much background instruction so, as an academic writer, one must assume that the reader has some level of knowledge. Yes, this makes the paper less accessible, but the point of the paper is not so much to educate as to inform. Education comes from books, courses and discussions with academics.

Until recently there hasn''t been much call for AI books for those outside of academia (other than general interest books). There are more titles on the shelves now and more in the future. Heck, maybe I should jump on the band wagon and put my PhD to some use... I''m sure it would enable me to sell a half dozen copies at least!




quote:
Original post by Geta
How would one go about implementing effective Plan Recognition and accurate Opponent Modelling for games like Age of Kings or Empire Earth which must operate in real time, with upwards of eight computer or human opponents, and hundreds (if not thousands) of agents to manage? And be able to process on a typical consumer system?



As usual, you''ve it the nail on the head! This is the ONE reason why most interesting/advanced AI techniques have not yet made it into the commercial arena: lack of processing power available for real-time analysis.

However, more and more graphics functions have been moved to the GPU, freeing up the CPU for AI... and machines are getting faster (although that doesn''t mean machines in homes are getting faster). What we can hope for in the short term are ''pared-down'' versions of some of the bigger algorithms. Opponent Modelling of a single player in real time is certainly achievable today. Plan Recognition techniques are also practical in certain domains.

Remember though, that techniques such of these don''t have to run once per frame as they are strategic level algorithms. Opponent Modelling can take place over minutes or even hours of gameplay; the same with Plan Recognition.

Tactical level algorithms are another story obviously. In this regime I believe that reactive, nonlinear behaviours built up from basic building blocks will lead to the most interesting behaviours in the short term. As hardware gets faster and memory gets cheaper it will be possible to do more in the shortened time frames of this regime (for example, you will eventually be able to run a couple of dozen of John Laird''s Quake bots on a single machine, rather than having a few connecting from external machines via the port interface).

Cheers,

Timkin

Share this post


Link to post
Share on other sites
RPGeezus    216
Predictor:
Very well said.

Timkin:

I have to disagree a bit. Many publications, when it comes to computing, are way over the top in their level of complexity. Once the onion has been peeled, so to speak, the ideas tend to be fairley straight forward.

Treating AI, and most other programming related topics, as if it were a science (which they are not) does nothing to further the ''cause''.

Often times, in and out of academia, people tend to make the ideas of others (I say others because no one in here invented ANN''s, or GA/GP/EP) sound WAY more complicated than they have to be.

At the same time, I can understand why someone may not want to write an easily digested paper. It is a lot of effort after all.

An old friend of mine was a professor at a local university. He had his PhD in math, and enjoyed being refered to as doctor. The problem was that this guy hated teaching, and only did it so that he could persue his research. He also hated writing papers, and again only did it because he was forced to. As a result, his classes stunk, his students hated him, and I''m sure his papers were nightmares to read. Still, he was very brilliant, and did amazing work in different computer related fields.

Cheers,
Will

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
Again, as the layman, and a little OT..
What''s GE and GP?
(I''m not asking for an entire description, just what the abbrevs stand for... I can google for the rest )

Share this post


Link to post
Share on other sites