Discussion on Squads & Group Behaviors

It has also been stated that it would make sense to incorporate environmental hints to point the AI to good areas for flanking or a good place to concentrate suppressive fire.

The AI of Killzone 2's Multiplayer Bots

The first part was held by Remco, who told us about the overall scope of the game. He told us that the bots in Killzone 1 were a well received feature and thus they wanted to focus more on bots for multiplayer games in Killzone 2. While the bots in Killzone 1 were more meant to accompany the player, the new bots had also the role of a teacher, telling the player how to play the game.

Focusing more onto the multiplayer part with 32 players in a game also allowed for more team based game modes. Per map multiple modes such as "Capture and Hold" (Domination), "Body Count" (Team Deathmatch), "Search and retrieve", "Search and destroy" and "Assassination" are available. Due to this, the AI had to adapt to the game mode on the map. Additionally, the AI had to handle badges adding capabilities dependent on the agents assigned character type (Scout, Tactician, Assault, Engineer, Medic or Saboteur).

While the previous version of the Killzone AI made usage of Lua, the new architecture is layered and split between Strategic AI, Squad AI and Individual AI. You can consider this as being a top down way of handling situations. The Strategic AI will hand orders to the squads like "Defend this point", "Advance to that point" or "Attack that unit". The squad will try to solve those orders and report to the strategic AI about the outcome of its operations.

The squad AI, while trying to obey to the strategic AI's orders, will order the individual AI to move to a specific location and receive feedback about combat information. The strategic AI may also assign targets to the individual AI and receive requests for assignments from it, but this strategic AI to individual AI communication seems not to be the "normal" way to handle this but more of a special case when the situation and game mode makes it necessary.

The individual AI makes use of hierarchical task network planning (HTN) after having created a "picture" of the current world state. This picture is created with daemons which collect orders, messages and threads (which have been recognized through stimuli and perception). The planner creates a plan which contains tasks which themselves create controller input (thus, the game AI makes usage of controller input to move around the NPCs).

Killzone 2's HTN planner creates one domain per individual AI. Each domain will contain one or more methods containing one or more branches with preconditions and task lists. A task can be a primitive such as "Shoot" or "Reload" or a compound which resolves recursively. Using this, a general combat behavior is created which is opportunistic and ordered.

Some numbers of the individual AI:

• 360 methods
• 1048 branches
• 138 behaviors
• 147 continue branches.

The squad AI makes usage of the same planning structure. The difference is that the daemons will receive strategic orders and the task execution will contain individual orders member messages which will then trigger individual AI planning as depicted above.

Alex then took over the presentation and reminded us of the challenge they encountered: 1-14 bots per side put into squads. He compared Killzone 2's AI with Halo 3: While Halo 3 has disposable bots, which are level specific and made to be easily handled by designers. The AI is mostly declarative and story driven. Killzone 2 on the other hand has persistent bots which have are applied in a more general way and have been programmed by programmers for programmers. It's mostly procedural and strategic.

As a reminder: The AI has to separate the "what" and the "how" in order to achieve good plans in a goal-driven approach. Killzone 2 did this my creating mission-specific AI (one for "Search and Destroy", one for "Capture and Hold", etc) containing sets of C++ classes. Also dependent on the game mode different objectives are generated and specific hints are set onto the maps to inform the AI where to find offensive and defensive locations.

Using these objectives a base strategy is created which is used to create tasks for the squad and individual AI. The objectives scale up and down with the number of bots which need rich and detailed information about the objective to reach. The objectives can be laid out this way:

1. Calculate ideal distribution of bots, then squads.
2. Create new squads if necessary.
3. Remove extra squads if too many assigned to any objective.
4. Pick an objective for each squad. This is based upon a weighting of each objective.
1. If the objective is already active, pick a new sub-objective on a regular basis.
2. Assign the best objective to each squad if the above is not valid.
5. Un-assign bots if too many for squad or objective.
6. Process all free bots and assign them to the best squad.

He then elaborated in a more detailed way the hints placed upon the map. The annotations are placed manually on the map ("create the information by hand first, then automate later if necessary") and include regroup locations, mission specific points to defend, sniping and hide locations. Using this information, the level is then automatically processed to create a strategic graph which helps support runtime strategic decision making and interpret the manual annotations dynamically.

The strategic map contains a set of areas which are groups of waypoints. These areas are connected and the number of connections already gives a hint about choke points in the level. The strategic map is used to make a rough squad movement planning (the actual waypoints are then used to do the actual path-finding). This high level overview of the map enables also an easy representation of faction influence on the map.

The influence map enables the AI to determine who controls which areas on the map. It is calculated using all bots, turrets and deaths. Through this information, the strategic planning can decide where to choose regrouping areas, where to attack, what the current progress is, etc. To select i.e. a regroup selection the strategic AI must identify candidates using recently used (by other squads) locations and previously selected locations. The influence map is also used to calculate movement on the map. Its information provides hints about areas appropriate for flanking or movement without getting into a fight.

Finally Alex told us about the future work which will include data mining to find ideal positions and hot spots. The usage of bots in early versions of the maps will help to improve the level design because this enables the designers to let the bots test run the levels and find weak points in their design. The bots role of a teacher will become more important. They will teach us how to play the game. In the future the interaction between a squad and other friend-squads and/or the human player will become more important, too. Currently a squad is not able to be of meaningful use to a player (from an automated point of view).

To learn more about the topic, see:

"Realistic Autonomous Navigation in Dynamic Environments" (Alex J. Champandard)
"HTN planning for flexible coordination of multiagent team behavior" (Obst, Mass, Boedecker)
"HTN planning: Complexity and Expressivity" (Erol, Hendler, Nau)
"Using Player Models to Improve Robustness of HTN Plans in Multi-Agent Domains" (Semsch, Jakob, Doubek, Pechoucek)
"Hierarchical Task Network Planning: Formalization, Analysism and Implementation" (Erol)

Page 3