codeliftsleep

Behavior Pure Decision AI...best method to use

Recommended Posts

I'm building an American football simulation(think football manager), and am wondering about the best way of implementing AI based on various inputs which are weighted based on the personality of the NPC...I have a version of Raymond Cattell's16PF model built into the game to be able to use their various personality traits to help guide decisions.

I am going to use this extensively so I need this to be both flexible and able to handle many different scenarios.

For instance, a GM has to be able to decide whether he wants to resign a veteran player for big dollars or try and replace them through the draft. They need to be able to have a coherent system for not only making a decision in a vacuum as a single decision but also making a decision as part of a "plan" as to how to build the team...For instance it makes no sense for a GM to make decisions that don't align with each other in terms of the big picture. I want to be able to have the decisions take in a wide range of variables/personality traits to come up with a decision. 

 

There is no NPC per se...There isn't going to be any animations connected to this, no shooting, following, etc...just decisions which trigger actions. In a situation like a draft, there is one team "on the clock" and 31 other teams behind the scenes working on trying to decide if they want to try and trade up, trade down, etc which can change based on things like who just got picked, the drop off between the highest graded player at their position/group and the next highest graded player in that position/next group, if a player lasts past a certain point, etc...

There needs to be all of these things going on simultaneous for all the teams, obviously the team on the clock is goifn to have to decide whether it wants to make a pick or take any of the offers to move down in the draft from other teams that might want to move up, etc..

So I am planning on making use of something called Behavior Bricks by Padaone Games( bb.padaonegames.com )which is a Behavior Tree but in conversations with others who have worked on AI in major projects like this(EA sports) they said to combine this with a State Machine.

My question is would I be able to do this using just Behavior Bricks or would I need to build the state machine separately?  Is there something else already created for this type of purpose that I could take advantage of?

Edited by codeliftsleep

Share this post


Link to post
Share on other sites

Ubiquitous answer for a starting point... 

http://intrinsicalgorithm.com/IAonAI/2012/11/ai-architectures-a-culinary-guide-gdmag-article/

Figure that everything you are going to be doing is weighing and scoring the relative value of different numerical inputs, biased by the personalities of the deciders. Straight up utility system is the way to go.

Share this post


Link to post
Share on other sites
29 minutes ago, IADaveMark said:

Ubiquitous answer for a starting point... 

http://intrinsicalgorithm.com/IAonAI/2012/11/ai-architectures-a-culinary-guide-gdmag-article/

Figure that everything you are going to be doing is weighing and scoring the relative value of different numerical inputs, biased by the personalities of the deciders. Straight up utility system is the way to go.

Awesome link, thanks!

How would I weigh immediate utility versus long term utility...ie, making a move that might make the team worse in the short run but will free up cap space and give them draft picks in return that they hope can make them better 2-3 years down the line?

 

Share this post


Link to post
Share on other sites

 

57 minutes ago, IADaveMark said:

Ubiquitous answer to all "how would I" questions...

If you were doing this in a pen-and-paper game or a board game, how would you make the decision?

Well, now that I think about it, probably I should do it the way real GMs do it...they evaluate the team against expectations at the end of the season and then make decisions as to what direction to take the team in.  I probably need to have a "Team State" which then would control the options the GM would consider while making decisions and would allow these decisions to align properly together...

Share this post


Link to post
Share on other sites

Coming at it from a different angle, there's the concept of 'discounted future rewards', which is a fancy way of saying "the chance of $1000 in a year's time is worth less than the same chance of $1000 today". You typically only want to make a sacrifice in the short term if the long term outcome is significantly better. So if you have some way to numerically measure (or, more likely, estimate) the quality or utility of a certain outcome, you will want to weight future outcomes less positively.

Share this post


Link to post
Share on other sites

Well that is not necessarily true for football. You have many things at play. Let's say you have a good player but you know he is going to demand top dollar and/or has told you he doesn't want to play there or won't resign there.

Will it make you better by trading this player today? No...but you will at least get some assets back in return instead of letting the player walk for nothing if you trade him now. There is no guarantee you will even get fair value(most times you won't) but something is better than nothing.

Sometimes a good player doesn't fit a scheme for a new coach/GM. He is a good player but not in this scheme. Trading him doesn't necessarily make you better short term, but if he can't do what you ask him to well there isn't much reason to keep him.

Maybe the player clashes with the new coach or GM, and he is going to be a problem...

There are also salary cap related decisions...You can't pay all your good players because you only have limited resources to spend or you end up screwing yourself over really badly for future years...For instance...a very good player wants to be paid like a Top 5 player in the league, but his true value isn't worth that...do you bite the bullet and overpay him or let some other team overpay him? Do you resign an aging vet with 1 more good year left to a 4 year contract knowing you are going to probably replace him with a younger cheaper player before the contract is over, and unless you can negotiate a low $ guaranteed money contract, are going to have a bunch of dead cap money floating around in future years or simply replace him now with a rookie who might not be as good this year but is a lot cheaper and who you expect can fill his shoes by year 2?

There are a myriad of reasons why a team will be willing to let a player walk in FA, not resign an FA, or trade a player for future picks and many times they KNOW they aren't getting fair value back, but do it anyway.

Football  reasoning/decision making is far more complex than these simple utility methods that work in shooter games because there are no guarantees you will be able to get more future value but you also have to be mindful of salary cap restrictions in the meantime and many other variables that come into play.

 

 

Edited by codeliftsleep

Share this post


Link to post
Share on other sites

Suggesting that the economic and strategic decision-making in football is different than the decision process used in FPS games is... uh... fairly obvious? I would suggest that it hardly bears mentioning. However, comparing it to something like a TBS game like Civ is more accurate and has plenty of overlap in the mentality and methodology. And yes, Civ-style games are almost entirely utility-based (i.e. mathematical modeling of decisions).

As for uncertainty of the payback, it's pretty easy to apply an estimated success rate of payoffs and multiply the payoff amount by that. e.g. I have a 75% chance of this panning out and if it does, I will net $1000 worth of value. Otherwise, it may be $200 worth of value. So what's your average payoff given those factors? That's why it is called "maximization of expected utility".

I know a book you can read that covers mathematical modeling of behavior in game AI. Can't think of the title right now, though...

Share this post


Link to post
Share on other sites

Replying to the original poster:

I think you misunderstood me. Pretty much any reward, when postponed to the future, has to be somewhat discounted, because it is not as good as the same reward received today. Things can change between now and the future which make the same reward less useful. So that's just an abstract way of weighing "immediate vs long term" utility, which is what was mentioned above.

In your specific example directly above you're actually comparing 2 quite different outcomes, and it's reasonable for them to be scored differently.

Everything you mention can be modelled with a utility-based system. I don't think a behaviour tree system is a good fit for this sort of problem at all, especially not when you want to factor in the personality scores. Behaviour trees are good for situations when there are is a clear hierarchy of prioritised choices and real-time tasks to be performed based on those choices.

A reasonable approach would be to try and model the outcomes from an 'average' point of view, and then to apply different weightings based on the personality factors you mention. For example, when you say "Will it make you better by trading this player today? No...but you will at least get some assets back in return" - you'd model the estimated benefit of those assets, and they would be compared to the estimated benefits of keeping the player. Those estimates can vary based on the personality of the person doing the assessment.

If you need to perform multiple decisions as part of an overall plan, there are a couple of approaches you'll want to consider:

  • actual planners - they are algorithms that run through various permutations of actions and work out which ones to do, and in which order. They usually assume there is a clear definition of success however, which you are unlikely to have.
  • state-space search (such as A*) - these algorithms are similar to planners but try to find the best route through 'state space'; again, this is usually about trying different hypothetical actions and measuring the expected outcomes.

 

Share this post


Link to post
Share on other sites

Well, one of the reasons I brought up state machines combined with Behavior Trees was from a response I got on here from a guy who worked with EA Sports designing AI for NCAA Football among other things, but I have quite a few questions regarding implementation that I'd need to understand before going down a route like that...plus as with anything dealing with programming there are almost always multiple ways to achieve the same end result so I thought I'd get other opinions and see which one would make more sense to me...

https://www.quora.com/What-is-the-best-option-for-a-Pure-Decision-AI-in-Unity

 

Share this post


Link to post
Share on other sites
24 minutes ago, Kylotan said:

The answer given there seems to refer primarily to in-game character control, which is a quite different problem from the long term strategy issue of building the team over time.

Well I can see very easily how I'd be able to use the Utility method to handle both...

If, say, a player has a decision to make about whether to accept a lower offer to stay where he is or take a higher offer to go to a bad team, for example, I could use various personality traits like loyalty, greed, ImportanceOfWinning, etc to basically add or subtract dollars from the two contract offers, which is sort of what actually happens in their minds...if they are loyal and like where they are, that is "worth X amount of dollars" to them...likewise if they are greedy and not loyal than the offer for more money is the one they will likely go with...

So I can see how the Utility method could be easily modified to work with both but I'm wondering if the same could be true for the method he discussed as well...or is it possible utility methods would be used inside the behavior trees?

Edited by codeliftsleep

Share this post


Link to post
Share on other sites

A behaviour tree is basically a large prioritised list of actions. It looks more complex than that, because the hierarchical structure makes it very easy to adjust the priorities for large groups of actions, and to make certain actions never run unless other actions have happened first, but ultimately it's a list based around some very binary decisions made at each branch of the tree. It's theoretically possible to inject utility scores in there, and this is commonly done for very specific sub-tasks, but it's not a practical approach on a wider scale.

Share this post


Link to post
Share on other sites

A "state machine" isn't necessarily a decision architecture. It is a way of keeping track of what state you are in. You can use all sorts of stuff on each of your many transitions to determine what to do next. A BT is, ultimately, going to result in you being in a state of some sort. The difference is that you are lifting the transition logic out of the individual states and putting them into a single reasoner. The result is the same, but the decision logic is cleaner. A utility system will also put you in a state if you want it, but, like a BT, the reasoner is a separate entity. 

For your game, the heavy lifting is done in the reasoner itself. You actually will have few, if any, "states" that you are in. You may have "mindsets" that coaches and GMs are in (e.g. "rebuilding year") but those are just numbers that affect how you look at other numbers.

 

Share this post


Link to post
Share on other sites
4 hours ago, IADaveMark said:

A "state machine" isn't necessarily a decision architecture. It is a way of keeping track of what state you are in. You can use all sorts of stuff on each of your many transitions to determine what to do next. A BT is, ultimately, going to result in you being in a state of some sort. The difference is that you are lifting the transition logic out of the individual states and putting them into a single reasoner. The result is the same, but the decision logic is cleaner. A utility system will also put you in a state if you want it, but, like a BT, the reasoner is a separate entity. 

For your game, the heavy lifting is done in the reasoner itself. You actually will have few, if any, "states" that you are in. You may have "mindsets" that coaches and GMs are in (e.g. "rebuilding year") but those are just numbers that affect how you look at other numbers.

 

OK got it...that makes more sense now...

Share this post


Link to post
Share on other sites

As for the "off year", that is a good utility function.

If all your options are to buy then that's all the AI will do.  The AI won't have an option to NOT buy because all the options available are to buy.  If you want the AI to have an option of not buying, you need to add those in as well.

The action could be something as simple as "don't buy". Or it could be "save for a bigger purchase", that could evaluate what is available right now versus speculation about what might be available later, and the costs of both. 

If you include additional options, "saving for the future", or "off year", or "nothing looks interesting", the new options provide an alternative to spending everything.  Each could have their own weighting functions, and there can be more than one of them.   A baseline "nothing looks interesting" is useful to have in nearly all scenarios, since you don't want the AI to spend every coin they've got.... unless you really do want them to spend everything.

 

Share this post


Link to post
Share on other sites

I would tack on to that by saying that having a true "nothing looks good" option is a must. You need to be able to distinguish between "I am deliberately choosing to do nothing" and "I am not doing anything because the AI is busted."

Share this post


Link to post
Share on other sites
1 hour ago, frob said:

As for the "off year", that is a good utility function.

If all your options are to buy then that's all the AI will do.  The AI won't have an option to NOT buy because all the options available are to buy.  If you want the AI to have an option of not buying, you need to add those in as well.

The action could be something as simple as "don't buy". Or it could be "save for a bigger purchase", that could evaluate what is available right now versus speculation about what might be available later, and the costs of both. 

If you include additional options, "saving for the future", or "off year", or "nothing looks interesting", the new options provide an alternative to spending everything.  Each could have their own weighting functions, and there can be more than one of them.   A baseline "nothing looks interesting" is useful to have in nearly all scenarios, since you don't want the AI to spend every coin they've got.... unless you really do want them to spend everything.

 

After more thinking about this, I actually am going to create enums with both team states and cap states because the two don'thave to be exclusive to each other...

 

For instance, the team states I have right now are TotalRebuild(What Jets and Browns did the last few years---complete firesale aiming for #1 pick in draft), Rebuild, Transition(when a team changes coaches/GM's and philosophies that go with it meaning players might be let go due to scheme fits), Building, WinNow and Reloading(when a team realizes the current key players aren't getting it done so they replace them but still expect to win(usually through FA))

Cap state will be tied to how they spend money under the cap and current options are CapConcious(sets a value or players and won't overspend regardless of how good they are), SpendOnOwn(team mostly focuses on resigning it's own payers they've drafted and very little in FA), SpendOnFA(teams focus on filling in holes via FA), and Flexble(meaning more or less they don't have a plan and do whatever---which is a lot of teams surprisingly--think Redskins)

Share this post


Link to post
Share on other sites
5 hours ago, codeliftsleep said:

OK got it...that makes more sense now...

If you read the "culinary guide" link I posted earlier, it has all of this in there and more. It will prevent you from intermixing your terms the way you are.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Announcements

  • Forum Statistics

    • Total Topics
      628354
    • Total Posts
      2982230
  • Similar Content

    • By koto
      Hello,

      I'm developing a roguelike game. Currently it's in a playable state and I think it's a high time to show it and hopefully get some feedback.

      The game has following features:
      - turn based
      - procedurally generated levels
      - tons of loot
      - spells system
      - trading system
      - some quests
      - some limited crafting
      - door/key/lever mechanics
      - quite a lot of different (look/behaviour) monsters

      Trailer:
      Demo: http://kotogames.com/wp-content/uploads/2017/11/OuaD_demo.zip
      Regards,
      Tom
       
    • By Afambus
      Hello,
      I have a full experienced team that looking for a composer and another 3d designer, im expecting as much from you there are other people in this server that can help you we have a determined team, we have done many things in our game there is more information in the discord about everyone and there skills there region and about the game, if you join the discord pm asap
      https://discord.gg/b2teN3m
    • By idzohar
      I am trying to rotate my car sprite just slightly to the left or right depending on input up to a small maximum. When I use this code, the rotation doesn't stop at the maximum i give it. When i try to reset the rotation back to its original state when not pressing any input, the car also jitters between a z axis rotation of 0 and 2.5. I don't understand why the rotation doesn't stop at the given maximums, nor do I understand why its jittering. Can anybody provide me some insight?
              float movementHorizontal = 0f;
              float movementVertical = 0f;
              if (Input.GetKey(KeyCode.A) && Input.GetKey(KeyCode.D))
              {
              }
              else if (Input.GetKey(KeyCode.A))
              {
                  movementHorizontal = -1;
                  if (transform.rotation.z < 10f)
                  {
                      transform.Rotate(zAxis, 2.5f);
                  }            
              }
              else if (Input.GetKey(KeyCode.D))
              {
                  movementHorizontal = 1;
                  if (transform.rotation.z > -10f)
                  {
                      transform.Rotate(zAxis, -2.5f);
                  }
                  
              }
              else
              {
                  if(transform.rotation.z > 0)
                  {
                      transform.Rotate(zAxis, -2.5f);
                      //transform.rotation.Set(0, 0, 0, 0);
                  }
                  else if (transform.rotation.z < 0)
                  {
                      transform.Rotate(zAxis, 2.5f);
                      //transform.rotation.Set(0, 0, 0, 0);
                  }
                  
              }
    • By BAG Labs
      Mobile SoS

      Platform: Android
      Genre: Board
      Link: Google Play
       
      This games sharpen memory and test your strategies to place S-O-S pattern within time limit and serve 3 difficulties as Easy, Normal, and Hard.

      Goals of the game is to put S-O-S words in patterns (Horizontal, Vertical, and Diagonal) alternately with enemy.
       
      Features:
      Single Player Multiplayer Achievements Leaderboards  
       
      Screenshot:

       

       

       
      Link: Google Play
       
      Please help us improve this game with review
    • By abarnes
      Hello!
      I am a game development student in my second year of a three year program and I would like to start building my portfolio. I was thinking of creating some games to show what I can do to potential employers since I wont have any work related experience when I graduate. But as I'm sure you all know there are tons of ways to approach developing/designing a game and I'm curious if anyone had any insight as to any "standards" that come with this? Is it okay to use game engines like Unity, Unreal, Game Maker etc? Or would it be better to make a game from scratch to better show case your skills? Any and all advice will be greatly appreciated!
  • Popular Now