Hi,

I'm currently programming the AI for enemies. The game uses an active-time based combat system; so combatants usually don't get an equal number of turns (like final fantasy). Each action that can be taken in combat has a preparation time, and a cooldown which determines when that entity gets to go again after it acts.
The following types of skills can be used in combat: offensive direct damage skills, skills that cause targets to be affected by positive or negative effects, and healig skills.

I'd like the AI to be able to do the following:
1. Attack intelligently; it should assess the threat posed by various player units and/or choose a target such that its attack does the most damage. However, I also need to be able to assign value to certain player targets so that more important targets can override the usual decision. There are also some skills that require prerequisite skills to be used first; so it needs to know how to use these skills, and somehow decide between lower damage attacks that requires no preparation or these more riskier attacks.
2. Use effects intelligently.
3. heal intelligently. Some allies may be more important than others, so it needs to assess the value of its allies.
4. Do all this while also factoring the length of time that various actions take.

How would I go about doing this? I've been reading up on tools used in AI like state machines, and behaviour trees, but i'm not sure how to apply them to this specific situation.

The various AI techniques have their pros and cons. Your question shows that your current problem is the reasoning, i.e. the selection of the best option under consideration of various factors.

The strength of both FSMs and BTs is IMO not reasoning. FSMs are less prominent today because they become unmaintainable quickly when the number of states and/or transition grow. However, reasoning in FSMs is implemented in the selection of one of the available transitions. Transitions are labeled with the condition that need to be met, and the conditions result in a boolean value. Strictly seen there must be exactly one transition showing a true. That makes the formulation of conditions for you very hard. Selection in a BT has many flavors. None of the standard ones solve your problem. BT selectors are IMO mainly good to control behavioral proceedings.

Now, another AI tool is utility. Utility based AI is good at reasoning like you want, because

* it is able to consider any amount and kind of factors for each option,

* looks at all options in parallel when deciding which one to select,

* is able to give multiple results in case that an option can e.g. be applied to multiple targets.

My favorite solution is a tree of selectors and actions, where each selector can be a utility-based reasoner, a BT node, a planner, or even a FSM, whatever matches the need at that specific level of AI. Let's say a top level utility-based reasoner has selected an option due to its best fitting utility value. The action sequence of the option is hence processed. If the behavior of the selected option can be modeled as sequence than all is fine. If not, then place e.g. a BT selector node (or even sub-tree) therein. Or, if the action need to be planned due to runtime circumstances, then place a planner therein.

However, implementing all this in entirety is much stuff, perhaps too much for your game. I recommend to start with a tree structure in mind and to add kinds of selectors and actions on need. A utility-based reasoner as first selector kind is obviously fine. Supporters for utility-based AI are around here, probably most notably our Dave himself. So don't hesitate to ask if you have more questions.

I would suggest playing through a few battles by hand writing out why you choose a particular option. This will show you what decisions your AI will need to be making and what data it will need to operate on. This should help you see how this might fit better to a state machine or behavior tree or a combination of both. As suggested by Haegarr the tools can be combined together.

Another vote for looking into utility systems for your problem. Utility systems are awesome.

Pay attention to the bits in the "Building a Better Centaur" presentation where they talk about escaping early and sorting considerations (I call them criteria - not sure what's a better fit yet - semantic wars). Not only do you get a really organic feel for cheap, but it gets even cheaper by being able to just say "meh ... I can't win ... screw this."

I also reuse the system to determine the weights for different "ActionSets," probably what Dave calls packages.

Handling targets can be a bit tricky to do cleanly though.

Fast track to all of Dave's (and co-conspirators) presentations:

https://www.google.com/search?q=GDC+Vault+Dave+Mark

If for whatever reason you find the curves terrifying here's a jump-start on the curves (C# from a Unity port of my C++ implementation of Dave's IAUS):

http://hastebin.com/useqoxumox.cs

As long as you write a GUI for manipulating the inputs then curves are a no brainer and tools like https://www.desmos.com/calculator are helpful, lately I use some pretty wacky curves.

In practice I have curves on the variables of a curve, curv-ception.

Hi,

I've progressed further on the combat implementation. Currently, I have skills set up as lists of affects that can occur to each type of target. For example,

"fireball" : {
"preparationTime" : 10,
"cooldownTime" : 5,
"enemy" : {
"damage" : 5,
"effects" : "stun",
"healthRestored" : 0
}

On a healing skill, it would be
"heal" : {
...
"self" : {
"damage" : 0,
"effects" : [],
"healthRestored" : 10

I'm looking to use a utility-based AI to evaluate which skills should be used. The problem is that skills can do anything, e.g damage, grant/remove effects, etc, so I've been trying to find a good utility function.

An idea I tried is a linear combination, all values normalized in the range [0, 1]:
utility (simplified) = danger * expectedHealing + (1 - danger) * expectedDamage

This doesn't give good results. I want the utility of using healing skills to be more like a quadratic curve, and that for damaging skills to be linear. How can I combine differing curves together so that the result is still normalized? Some skills can do various things, like do damage and cause effects, so the utility from effects needs to be combined too. Is there a better way of evaluating utility in this setup?

Ah, this sounds more like alpha-beta search where the utility is the happiness of the agent at some possible future state, instead of the utility computed on the changes that an action would cause to the game state, if that makes any sense.

I would first model the effects of the action on the game state (health, mana, whatever), and then write the utility function describing how happy I am with the game state. If the result of an action is random, you should pick the action whose expected utility is maximum.

Instrument your code so it can write a log file describing why a decision was taken. Using that you should be able to tweak your utility function whenever you are unhappy with a decision by an agent. If you have good mathematical intuition and a bit of common sense, you should be able to make it work really well.

If you find it hard to come up with a decent utility function on your own, perhaps you should read Dave Mark's book.