codeliftsleep

Behavior Pure Decision AI...best method to use

Recommended Posts

I'm building an American football simulation(think football manager), and am wondering about the best way of implementing AI based on various inputs which are weighted based on the personality of the NPC...I have a version of Raymond Cattell's16PF model built into the game to be able to use their various personality traits to help guide decisions.

I am going to use this extensively so I need this to be both flexible and able to handle many different scenarios.

For instance, a GM has to be able to decide whether he wants to resign a veteran player for big dollars or try and replace them through the draft. They need to be able to have a coherent system for not only making a decision in a vacuum as a single decision but also making a decision as part of a "plan" as to how to build the team...For instance it makes no sense for a GM to make decisions that don't align with each other in terms of the big picture. I want to be able to have the decisions take in a wide range of variables/personality traits to come up with a decision. 

 

There is no NPC per se...There isn't going to be any animations connected to this, no shooting, following, etc...just decisions which trigger actions. In a situation like a draft, there is one team "on the clock" and 31 other teams behind the scenes working on trying to decide if they want to try and trade up, trade down, etc which can change based on things like who just got picked, the drop off between the highest graded player at their position/group and the next highest graded player in that position/next group, if a player lasts past a certain point, etc...

There needs to be all of these things going on simultaneous for all the teams, obviously the team on the clock is goifn to have to decide whether it wants to make a pick or take any of the offers to move down in the draft from other teams that might want to move up, etc..

So I am planning on making use of something called Behavior Bricks by Padaone Games( bb.padaonegames.com )which is a Behavior Tree but in conversations with others who have worked on AI in major projects like this(EA sports) they said to combine this with a State Machine.

My question is would I be able to do this using just Behavior Bricks or would I need to build the state machine separately?  Is there something else already created for this type of purpose that I could take advantage of?

Edited by codeliftsleep

Share this post


Link to post
Share on other sites
IADaveMark    3741

Ubiquitous answer for a starting point... 

http://intrinsicalgorithm.com/IAonAI/2012/11/ai-architectures-a-culinary-guide-gdmag-article/

Figure that everything you are going to be doing is weighing and scoring the relative value of different numerical inputs, biased by the personalities of the deciders. Straight up utility system is the way to go.

Share this post


Link to post
Share on other sites
29 minutes ago, IADaveMark said:

Ubiquitous answer for a starting point... 

http://intrinsicalgorithm.com/IAonAI/2012/11/ai-architectures-a-culinary-guide-gdmag-article/

Figure that everything you are going to be doing is weighing and scoring the relative value of different numerical inputs, biased by the personalities of the deciders. Straight up utility system is the way to go.

Awesome link, thanks!

How would I weigh immediate utility versus long term utility...ie, making a move that might make the team worse in the short run but will free up cap space and give them draft picks in return that they hope can make them better 2-3 years down the line?

 

Share this post


Link to post
Share on other sites

 

57 minutes ago, IADaveMark said:

Ubiquitous answer to all "how would I" questions...

If you were doing this in a pen-and-paper game or a board game, how would you make the decision?

Well, now that I think about it, probably I should do it the way real GMs do it...they evaluate the team against expectations at the end of the season and then make decisions as to what direction to take the team in.  I probably need to have a "Team State" which then would control the options the GM would consider while making decisions and would allow these decisions to align properly together...

Share this post


Link to post
Share on other sites
Kylotan    10010

Coming at it from a different angle, there's the concept of 'discounted future rewards', which is a fancy way of saying "the chance of $1000 in a year's time is worth less than the same chance of $1000 today". You typically only want to make a sacrifice in the short term if the long term outcome is significantly better. So if you have some way to numerically measure (or, more likely, estimate) the quality or utility of a certain outcome, you will want to weight future outcomes less positively.

Share this post


Link to post
Share on other sites

Well that is not necessarily true for football. You have many things at play. Let's say you have a good player but you know he is going to demand top dollar and/or has told you he doesn't want to play there or won't resign there.

Will it make you better by trading this player today? No...but you will at least get some assets back in return instead of letting the player walk for nothing if you trade him now. There is no guarantee you will even get fair value(most times you won't) but something is better than nothing.

Sometimes a good player doesn't fit a scheme for a new coach/GM. He is a good player but not in this scheme. Trading him doesn't necessarily make you better short term, but if he can't do what you ask him to well there isn't much reason to keep him.

Maybe the player clashes with the new coach or GM, and he is going to be a problem...

There are also salary cap related decisions...You can't pay all your good players because you only have limited resources to spend or you end up screwing yourself over really badly for future years...For instance...a very good player wants to be paid like a Top 5 player in the league, but his true value isn't worth that...do you bite the bullet and overpay him or let some other team overpay him? Do you resign an aging vet with 1 more good year left to a 4 year contract knowing you are going to probably replace him with a younger cheaper player before the contract is over, and unless you can negotiate a low $ guaranteed money contract, are going to have a bunch of dead cap money floating around in future years or simply replace him now with a rookie who might not be as good this year but is a lot cheaper and who you expect can fill his shoes by year 2?

There are a myriad of reasons why a team will be willing to let a player walk in FA, not resign an FA, or trade a player for future picks and many times they KNOW they aren't getting fair value back, but do it anyway.

Football  reasoning/decision making is far more complex than these simple utility methods that work in shooter games because there are no guarantees you will be able to get more future value but you also have to be mindful of salary cap restrictions in the meantime and many other variables that come into play.

 

 

Edited by codeliftsleep

Share this post


Link to post
Share on other sites
IADaveMark    3741

Suggesting that the economic and strategic decision-making in football is different than the decision process used in FPS games is... uh... fairly obvious? I would suggest that it hardly bears mentioning. However, comparing it to something like a TBS game like Civ is more accurate and has plenty of overlap in the mentality and methodology. And yes, Civ-style games are almost entirely utility-based (i.e. mathematical modeling of decisions).

As for uncertainty of the payback, it's pretty easy to apply an estimated success rate of payoffs and multiply the payoff amount by that. e.g. I have a 75% chance of this panning out and if it does, I will net $1000 worth of value. Otherwise, it may be $200 worth of value. So what's your average payoff given those factors? That's why it is called "maximization of expected utility".

I know a book you can read that covers mathematical modeling of behavior in game AI. Can't think of the title right now, though...

Share this post


Link to post
Share on other sites
Kylotan    10010

Replying to the original poster:

I think you misunderstood me. Pretty much any reward, when postponed to the future, has to be somewhat discounted, because it is not as good as the same reward received today. Things can change between now and the future which make the same reward less useful. So that's just an abstract way of weighing "immediate vs long term" utility, which is what was mentioned above.

In your specific example directly above you're actually comparing 2 quite different outcomes, and it's reasonable for them to be scored differently.

Everything you mention can be modelled with a utility-based system. I don't think a behaviour tree system is a good fit for this sort of problem at all, especially not when you want to factor in the personality scores. Behaviour trees are good for situations when there are is a clear hierarchy of prioritised choices and real-time tasks to be performed based on those choices.

A reasonable approach would be to try and model the outcomes from an 'average' point of view, and then to apply different weightings based on the personality factors you mention. For example, when you say "Will it make you better by trading this player today? No...but you will at least get some assets back in return" - you'd model the estimated benefit of those assets, and they would be compared to the estimated benefits of keeping the player. Those estimates can vary based on the personality of the person doing the assessment.

If you need to perform multiple decisions as part of an overall plan, there are a couple of approaches you'll want to consider:

  • actual planners - they are algorithms that run through various permutations of actions and work out which ones to do, and in which order. They usually assume there is a clear definition of success however, which you are unlikely to have.
  • state-space search (such as A*) - these algorithms are similar to planners but try to find the best route through 'state space'; again, this is usually about trying different hypothetical actions and measuring the expected outcomes.

 

Share this post


Link to post
Share on other sites

Well, one of the reasons I brought up state machines combined with Behavior Trees was from a response I got on here from a guy who worked with EA Sports designing AI for NCAA Football among other things, but I have quite a few questions regarding implementation that I'd need to understand before going down a route like that...plus as with anything dealing with programming there are almost always multiple ways to achieve the same end result so I thought I'd get other opinions and see which one would make more sense to me...

https://www.quora.com/What-is-the-best-option-for-a-Pure-Decision-AI-in-Unity

 

Share this post


Link to post
Share on other sites
Kylotan    10010

The answer given there seems to refer primarily to in-game character control, which is a quite different problem from the long term strategy issue of building the team over time.

Share this post


Link to post
Share on other sites
24 minutes ago, Kylotan said:

The answer given there seems to refer primarily to in-game character control, which is a quite different problem from the long term strategy issue of building the team over time.

Well I can see very easily how I'd be able to use the Utility method to handle both...

If, say, a player has a decision to make about whether to accept a lower offer to stay where he is or take a higher offer to go to a bad team, for example, I could use various personality traits like loyalty, greed, ImportanceOfWinning, etc to basically add or subtract dollars from the two contract offers, which is sort of what actually happens in their minds...if they are loyal and like where they are, that is "worth X amount of dollars" to them...likewise if they are greedy and not loyal than the offer for more money is the one they will likely go with...

So I can see how the Utility method could be easily modified to work with both but I'm wondering if the same could be true for the method he discussed as well...or is it possible utility methods would be used inside the behavior trees?

Edited by codeliftsleep

Share this post


Link to post
Share on other sites
Kylotan    10010

A behaviour tree is basically a large prioritised list of actions. It looks more complex than that, because the hierarchical structure makes it very easy to adjust the priorities for large groups of actions, and to make certain actions never run unless other actions have happened first, but ultimately it's a list based around some very binary decisions made at each branch of the tree. It's theoretically possible to inject utility scores in there, and this is commonly done for very specific sub-tasks, but it's not a practical approach on a wider scale.

Share this post


Link to post
Share on other sites
IADaveMark    3741

A "state machine" isn't necessarily a decision architecture. It is a way of keeping track of what state you are in. You can use all sorts of stuff on each of your many transitions to determine what to do next. A BT is, ultimately, going to result in you being in a state of some sort. The difference is that you are lifting the transition logic out of the individual states and putting them into a single reasoner. The result is the same, but the decision logic is cleaner. A utility system will also put you in a state if you want it, but, like a BT, the reasoner is a separate entity. 

For your game, the heavy lifting is done in the reasoner itself. You actually will have few, if any, "states" that you are in. You may have "mindsets" that coaches and GMs are in (e.g. "rebuilding year") but those are just numbers that affect how you look at other numbers.

 

Share this post


Link to post
Share on other sites
4 hours ago, IADaveMark said:

A "state machine" isn't necessarily a decision architecture. It is a way of keeping track of what state you are in. You can use all sorts of stuff on each of your many transitions to determine what to do next. A BT is, ultimately, going to result in you being in a state of some sort. The difference is that you are lifting the transition logic out of the individual states and putting them into a single reasoner. The result is the same, but the decision logic is cleaner. A utility system will also put you in a state if you want it, but, like a BT, the reasoner is a separate entity. 

For your game, the heavy lifting is done in the reasoner itself. You actually will have few, if any, "states" that you are in. You may have "mindsets" that coaches and GMs are in (e.g. "rebuilding year") but those are just numbers that affect how you look at other numbers.

 

OK got it...that makes more sense now...

Share this post


Link to post
Share on other sites
frob    44974

As for the "off year", that is a good utility function.

If all your options are to buy then that's all the AI will do.  The AI won't have an option to NOT buy because all the options available are to buy.  If you want the AI to have an option of not buying, you need to add those in as well.

The action could be something as simple as "don't buy". Or it could be "save for a bigger purchase", that could evaluate what is available right now versus speculation about what might be available later, and the costs of both. 

If you include additional options, "saving for the future", or "off year", or "nothing looks interesting", the new options provide an alternative to spending everything.  Each could have their own weighting functions, and there can be more than one of them.   A baseline "nothing looks interesting" is useful to have in nearly all scenarios, since you don't want the AI to spend every coin they've got.... unless you really do want them to spend everything.

 

Share this post


Link to post
Share on other sites
ApochPiQ    23064

I would tack on to that by saying that having a true "nothing looks good" option is a must. You need to be able to distinguish between "I am deliberately choosing to do nothing" and "I am not doing anything because the AI is busted."

Share this post


Link to post
Share on other sites
1 hour ago, frob said:

As for the "off year", that is a good utility function.

If all your options are to buy then that's all the AI will do.  The AI won't have an option to NOT buy because all the options available are to buy.  If you want the AI to have an option of not buying, you need to add those in as well.

The action could be something as simple as "don't buy". Or it could be "save for a bigger purchase", that could evaluate what is available right now versus speculation about what might be available later, and the costs of both. 

If you include additional options, "saving for the future", or "off year", or "nothing looks interesting", the new options provide an alternative to spending everything.  Each could have their own weighting functions, and there can be more than one of them.   A baseline "nothing looks interesting" is useful to have in nearly all scenarios, since you don't want the AI to spend every coin they've got.... unless you really do want them to spend everything.

 

After more thinking about this, I actually am going to create enums with both team states and cap states because the two don'thave to be exclusive to each other...

 

For instance, the team states I have right now are TotalRebuild(What Jets and Browns did the last few years---complete firesale aiming for #1 pick in draft), Rebuild, Transition(when a team changes coaches/GM's and philosophies that go with it meaning players might be let go due to scheme fits), Building, WinNow and Reloading(when a team realizes the current key players aren't getting it done so they replace them but still expect to win(usually through FA))

Cap state will be tied to how they spend money under the cap and current options are CapConcious(sets a value or players and won't overspend regardless of how good they are), SpendOnOwn(team mostly focuses on resigning it's own payers they've drafted and very little in FA), SpendOnFA(teams focus on filling in holes via FA), and Flexble(meaning more or less they don't have a plan and do whatever---which is a lot of teams surprisingly--think Redskins)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Similar Content

    • By menyo
      I'm currently testing out behaviortree's with the LibGDX AI library. I have read a lot about behaviortree's past couple days but it's hard to get clear how I need to build the tree for my specific scenario. I'm looking to use a behaviortree for a Rimworld like game where the players units are not directly controllable. In the first place I'm wondering if I should have a single big tree for the complete AI or many smaller tree's, for example a separate tree for:
      Moving an item Building a building Crafting an item Resting when sleepy Eating when hungry In the examples I have seen they all talk about a "single job". Like entering a building, GoTo -> Open Door -> GoTo -> Close door. But what if I need to check if I have the keys on me? And I need to check a lot of these variables.
      When A unit is Idle I'd like him to maintain his primary needs if he has access to them. If his needs are satisfied enough he can take on certain jobs like building walls or crafting items. I have a a lot of different jobs but jobs like building or crafting items are relatively the same with a different outcome so I could probably make a abstract job for that, it helps but I will still end up with a really huge tree though.
      Another issue I'm facing is that when tasks are running, and something more important pops up (enemy spotted or some kind of emergence task) the unit should stop it's current task and act accordingly to the interruption. So since the task is running I need to do those checks on each runnable task then returned failed/cancelled and further down the sequence I need to do another check for these interruptions and handle them accordingly. I have briefly read into dynamic branches, not sure if GDX AI supports this but adding a behavior to the tree to handle an interruption seems a good idea. These dynamic branches also opens the opportunity to hold behaviors at the jobs and once a unit accepts a job it inserts that branch into it's own tree.
      I hope I'm clear, it's hard to explain and get a global view of a complex behavior tree. I have read several times that behavior tree's are very easy to understand and implement. Well, that might be the case for those small tree's I find everywhere. On the other hand I might be over complicating things.
    • By WinterDragon
      My first goal is to build in python a text-based (next version will have graphics) adventure game scenario where a player enters a tavern, and goes on a quest. Earns money, buys weapons and gets in a fight.
       
      So I've got four ideas for games I want to build.
      1. a space simulation where a mute evolving clique is tasked with colonising planets - it's also a social network.
      My next goal is to build a 3rd person shooter and streets of rage style framework for a action/adventure game based on my story which is about God's mightiest heroes (with mutation sandbox) versus the devil (read pinhead meets bizarro superman).
      I also want to build in python, a 3d isometric rpg/sim that looks like the sims 1 but thematically is more like Bully or GTA1, but plays like Hell's Kitchen DS combined with Dofus, with environments combining elements of Phantasy Star, Sonic and art photography of convenience stores, etc.
      I also want to build an epic adventure story in RPGMakerMV.
      Any advice as to what else I should build in between to get closer to my goal projects.
       
      In a few weeks I'll have my first game, I'm not really sure how to turn it into an executable file so i can share it with people who don't have python installed.
    • By Steve Vermeulen
      I've been using async-await in Unity 2017 for awhile now and went through some hurdles getting it to work properly with the unity way of doing things. In order to avoid anyone else suffering the same problems I've posted a tutorial with what I've learned here:

      http://www.stevevermeulen.com/index.php/2017/09/23/using-async-await-in-unity3d-2017/
    • By ReStart Studio
       
      Hi, I would like to have an opinion on what I can improve and change in my first published game.
      https://play.google.com/store/apps/details?id=com.restart.bouncy
      The game was made on Unity3D.
      Thank you.
    • By ilovegames
      You are a US Army soldier, and one day an unknown enemy attacked your base. Now your task is to survive and protect the base. There are a lot of weapons in your arsenal. Survive this nightmare at any cost. 
      Download https://falcoware.com/Defender.php
       



  • Popular Now