I've wondered about whether it might make sense to do some AI coding in something like LUA instead fo just C++. However I would think LUA is of most use if you just pass in a small amount of parameters - wheras AI code in my RTS will need to look at the map etc. Is this the kind of thing it is or is not useful for?

What you can do is expose a lot of your content through small accessor and mutator functions of the engine, and then use those functions from within LUA, rather than passing around large chunks of raw data to and from the scripting engine (which is very slow).

Is LUA quick to call C functions? Is this something that would actually aid development or would I just be doing it for the sake of it?
I'm thinking of maybe implementing a state machine with a class as each state etc - is LUA actually going to help? Is another language better?

You mean C DLLs to allow functionality to be dropped in?

That's why I'm asking about something like LUA - will it actually save any time apart from re-compiling my code?

I wouldn't. The AI code is alot about logic and such, yes, but it'll probably also want to do lots of calculations, and quite frequent data access calls to your engine and gameplay code. It may seem fine at first, but you'll end up in a place where you're "almost done" but have to rewrite all your AI since it can't be properly optimized.

So when IS it feasible to use such a tool? Level scripting?

That depends on your project, of course. Level scripting is an excellent example of where you might want to use some scripting language - but then again, maybe not. Generally, in FPS-style games, you want to have visual scripting for your designers (i.e. they dont write script code, they place boxes and connect lines between items).

From a technical standpoint, I love scripting. I've used lua to implement basicly everything in a hovercraft racing game - we had lua reading of map files, lua scripts as game modes (racing mode, collect items mode, whatever you want...), and lua for environment modifiers. There's lots of fun stuff you can do with such a system... you can enable sending of game modes over the net, and that sort of thing.

However, in your average "real" game project, I'd say scripting is of rather limited uses - moving code to scripts tend to introduce maintenance and debugging problems, while not adding much in the long term. We've basicly just got rid of our last python scripts after our gameplay programmers had been in a state of total hate for them because of the debugging problems.

In the end, it comes down a bit to mods and such - what part of your program do you expect "outsiders" to modify? Those are the prime candidates for a scripting language (unless you have a level editing program or such).

One area I'd forgotten about is high-level / planning AI - like the opponent AI in an RTS deciding what buildings to build, where to attack you etc. One issue here is that you want the main algorithm to be running continually but don't probably want the whole thing to run every frame - it may be too slow and more importantly the AI may think too quickly. One option is to do it as loads of states, but if it were running on a pseudo-CPU could you not restrict it to only a limited number of 'cycles' each frame?