Archived

This topic is now archived and is closed to further replies.

Sandman

.....Of Lieutenants and Learning Agents (RTS)

Recommended Posts

I was doing more thinking about control and unit coordination the other day, when I was struck by this stunning idea.... What if the commander could train his army himself? My first thought, was to have a computer controlled lieutenant in charge of your army, but subordinate to the player. However, rather than being a mere management agent for the player, this lieutenant actually learns the players tactics. This means that the player can concentrate on strategy without the need to micromanage his army. Of course, you still have complete control over everything, but micromanagement is massively reduced. It also gives rise to some more interesting ideas.... 1. Training missions: Of course, you have to actually train your lieutenant. You could set up war games, in which the army fights drones under your control. They also allow you to teach your tactics... 2. Set pieces/manouevres: You could associate certain signals to certain manouevres. Signals could be sent by hitting a key or icon, and could thus trigger a complete redeployment of your army or a drastic change of tactics. You could teach the computer to retreat when you hit R for example, and the Lieutenant would make sure your units all get out safely, while long ranged units give covering fire, for example. 3. Story: What if your Lieutenant defects halfway through the game? Not only do you have to teach your tricks to his replacement, but you have to learn to combat your own tactics.... Of course, the technical issues are great, but I dont think it is out of reach - Black and White showed us this. I think this might result in a very different game from a conventional RTS. Your thoughts, as ever, are welcome.

Share this post


Link to post
Share on other sites
WHAT IF THE COMMANDER COULD TRAIN HIS ARMY HIMSELF?

Love it. I''ve put this same idea into my own design (on paper). It''s exactly like you say: it takes away from the micro management process. And at the same time it adds to the realism factor. I mean, if the player is the general, he wouldn''t be busy teaching new recruits how to hold a sword and how to move across the battlefield. No, he would tell his commanders ''show those new recruits how to hold a sword and how to move across the battlefield.''

I like the thought of the commander(s) actually ''learn'' the player''s tactics. If the player uses a lot of charges, his commander will make sure to focus on that aspect of training with his units. If the player likes to ambush opponents, the commander will make sure to teach the units the finer arts of stealth.

Of course, the player should also be able to give very direct orders to the commanders. In other words, the player SHOULD be able to deal with the micromanagement if he or she wants to (and to whatever level he or she desires) If the general wants those cavalry units to learn how to shoot a bow and forego their sword practice, so be it. A good, strong commander might inform the general (player) that his latest orders might not be all that smart (''General, do you think that''s wise?'').

1) Training missions

In my own design the player can either recruit a commander and just let the computer take care of everything, or the player can take one unit on several missions and create a commander out of that unit. The first missions would show the single unit some of the different aspects of combat. Later missions would put the unit in control of a small squad of soldiers. The last stage would have the ''commander'' leading a large group of units on a mission, learning the many different tactics used in large scale combat.

2) Set pieces/maneuvres

I''m very fond of the flags that were used to issue orders in battles in the past. You know, general and some of his best commanders stand on the hillside, overlooking the battle. When they want to give an order, they have someone wave certain flags and the commanders fighting below will spot the signals and relay those orders to their troops.

You could still hit your ''R'' button for an all-out retreat. What this would do is:

1) have general issue ''retreat'' order to unit that is responsible for waving flags
2) have that unit wave the appropriate flag
3) have the commanders on the battlefield spot the flag order
4) have the commanders on the battlefield issue the ''retreat'' order
5) have the units on the battlefield follow the ''retreat'' order
(in a previously designed method)

All these steps would take a little bit of time. For example, it could take a full minute before the units on the battlefield finally learn of the ''retreat'' order.

A good communication between general and his commanders is of the utmost importance because of this.

One aspect that these flags might add is an actual use of the ''intelligence'' of the commanders. A smart commander might be able to learn 20 different flag orders... but if his units only learned how to act on 10 of those, then those other 10 are really of no use. You can''t have the commander tell his units ''okay... now, we''re going to... I mean... This is the plan...''
No, you''d have to establish how many commands each unit can learn and base your general commands on that. For example, the dumbest unit might be able to learn only 3 different orders: attack, defend and retreat. The smartest unit might be able to learn an extensive number of orders, varying from ''attack using method X in formation 3'' to ''retreat to a previously set location, moving with speed Y, covering units 4 and 5 with ranged fire and targeting mainly enemy units Z''.

It''s in the player''s best interest to know which units can do what. And it''s up to the player to assign his commanders to take advantage of the strengths and to fix the weaknesses of his units.

3) Story

Yup. If the commander that the player has been using extensively all of a sudden dies... he''ll have to adjust his tactics. I''ve always liked this, because it is a variant form of permanent death. The death of a commander would have a very high impact on the gameplay, but... it wouldn''t be the end. The player would be able to build up his forces again using a new commander. But it would take time.

If a player manages to create the strongest army around and decides to have some fun killing of a weaker army... he might lose his commander in the process. Suddenly, that strongest army is somewhat leaderless. Units function less efficient.

------

My only concern with this is that it would require a lot of computer power to give each commander/unit its own AI. Especially when the plan is to create large battles with many units.

Share this post


Link to post
Share on other sites
quote:

My only concern with this is that it would require a lot of computer power to give each commander/unit its own AI. Especially when the plan is to create large battles with many units.



Yes. It did occur to me to make each unit a learning agent in its own right, which would be extremely cool, but I think the resources required for this would be too much. Instead you have a single AI, a computer sidekick which may or may not have an actual presence on the battlefield.

As for the signals, it is very similar to the flag approach except the flags are undefined at the start of the game - only by training your army to understand the retreat signal will they respond to it.

It seems to me that our ideas differ in that I see this approach as an aid to normal control, ie control is essentially the same as any other RTS but you can train your commander to take some of the burden off you, whereas you seem to see it as a control mechanism in its own right. I think either approach would be very interesting.

An example of how starcraft might play with a trained commander:

Start the game. Player takes a drone off exploring while the commander automates the standard quick start for zerg, as you have trained it to do so.
After exploring the map for a while, the player finds the enemy base. His drone is getting wasted, so he hits a key to signal a standard "Scramble" order, despatching a group of hydras to defend the drone. Shortly afterwards the player hits another key to signal to the lieutenant to start mass building zerglings.
Enemy reinforcements arrive, so he hits the retreat signal. The zerg units pull out. After a short wait, when the players hydras are healed and the zerglings are ready, he hits another key which signals a full offensive. As trained, the commander moves the ranged units, (hydras) to higher ground where they are safe from melee, and the hoardes of zerglings move in to attack, as well as diverting fire from the hydras. Soon the terran reinforcements are expended, and the base is destroyed....

The player is freed from the hassle of micromanagement, yet he still has to learn it to teach it to the computer. Similarly he is freed from the hassles of managing each units, and can concentrate on strategy, rather than tactics.

Edited by - Sandman on October 22, 2001 10:46:07 AM

Share this post


Link to post
Share on other sites
You could allow each unit to interface with the AI system, at a simpler level. For instance, The commander needs lots of resources to function effectivly, but each unit does not. So, if you could operate the AI at a simplified level (with n times less computation per descision), you could let the units themselves tap into the brain waves of thier commanders, and not have to learn everything themselves.

As for the flag thing, simple delayed messages would make it easy (Deliver "Retreat" to "all units" at "250 ticks in the future", would delay the messages for 250 game update ticks).

Also, commanders could OWN the units under thier control, and, while other commanders may give that unit an order, they will automatically check with thier commander to see if it''s ok. This is a promising idea. I love it!

Z.

Share this post


Link to post
Share on other sites
Another promissing thing, would be this...

Consider an outpost, that might be attacked by enemies. You can''t scramble an entire army to go there, so you just send your best Field Commander with his staff. They immediatly boost morale, and the longer they stay, the better the defenders become

Share this post


Link to post
Share on other sites
Heres a thought:

Why don''t you have the whole keyboard able to be set up to binds, and have all your major functions on the F Keys. So like, F1 would be help and F2 would be load the save menu, or something like that.

But now you have certain maneoveurs scheduled to different keyboard keys on the players decision.

For example:
Tell your commander to train your troops:
Hit Q: Show your troops the standard retreat movement.

Tell your commander to train your troops:
Hit W: Show your troops a retreat, where 10 units stay behind as bait.

Tell your commander to train your troops:
Hit E: Show your troops a retreat, where one group heads left and the other group heads right.

This prevents the player from being limited to certain amount of maneuevers, as they effectively have roughly 35 (alpha keys, + some) to have different maneuvers set up, and they can macro the keys according to their own preference in a way they will remember.

Just a thought.

Share this post


Link to post
Share on other sites
Estauns: That is pretty much exactly what I was aiming at... I think you could use all the lettered keys for signals (only 26, but that is still a fair few manouevres. You could have more if you allowed SHIFT/CTRL/ALT combinations, giving you as many as 78 possible commands, although this is probably far more than any player is likely to come up with and remember...) and keeping the numbered keys for unit assignment (I dont want to overcomplicate things for players who want/need to take control themselves, so the standard RTS controls should apply as much as possible)

Harder to implement would be teaching the computer to respond to the enemy on its own... eg how to respond when being flanked, or attacked in a pincer movement... or what different units should do when attacked...

Share this post


Link to post
Share on other sites
You should onyl have to train your commanders, as your troops should be trained in current tactics. So, when you want to create a new action, your commanders first hvae to learn it (by looking at maps, etc), and then training the troops. If you implemented this, there wouldn''t be any more "idle" time for troops, as they would always be marching around training =). Also, since troops are begin trained in current tactics, the more actions the player has bound, the longer it takes training (unless you "rush" the training, something more to add to the RTS toolbox =).

Z.

Share this post


Link to post
Share on other sites
I thought that your idea is what genetic algorithms is about. Where agents that learn a skill pass on this skill to its children. If this is the case, its not "new." All this elaborate "training" you want to do can be done by simply passing on the "genes" of the Lieutenants to his men in training, or only some of his genes, so they are not as perfect. Now you just gotta figure out how to implement AI into your game.

Share this post


Link to post
Share on other sites
GalaxyQuest:

Actually it is a lot simpler than you seem to be suggesting. Basically, the "commander" (which may have an avatar, but it may simply be a concept) acts a bit like a computer player. At the beginning of the game, it is a very stupid computer player, which does very little. The idea of the commander is not to win the game for you, but to take away a lot of the grunt work in coordinating your strategy. By teaching it tactics , you as a human player are left free to focus on the more important things, like strategy

As an example of what I am getting at, have you ever tried launching a massive Zerg rush on a computer opponent? StarCraft only allows you to control 12 units at a time, so a zerg rush consisting of 100 zerglings is actually quite difficult to implement. On some maps it is nearly impossible, as your zerg get split up by the scenery and end up arriving in ineffectual dribs and drabs, which are easily eradicated. The computer on the other hand, can execute these rushes flawlessly, since it is fast enough to babysit each unit individually, preventing it from getting lost, and easily attack you from four different directions at once. Even while simultaneously coordinating an attack from the air.

What I want to do is give the player a computer assistant that he can use to help him coordinate complex tactical manouevres (who would have thought a zerg rush would ever be considered a complex tactical manouevre? ) But how does the computer player know what you want him to do? You teach it, by showing it various manouevres.

The idea of individual units learning is great, and I would love to do that, but I really dont think that it is doable just yet, not unless your army is really small.

Zaei: Thats pretty much it, although you will only have one commander. It is more of a concept than an actual game unit..

Edited by - Sandman on October 24, 2001 6:10:24 AM

Share this post


Link to post
Share on other sites
I''d love to see this implemented in a campaign mode.

My personal design has the player take control over a small army and lead it through victories and defeats, much like players now control single characters in rpg games.

With a ''smart'' commander, you would really be able to teach the commander all the player''s favorite moves. After ten battles or so, the commander will have noticed that the player retreats if his troops are X % dead. The commander might anticipate this and warn the player that X % of the units have died. ''Do you want to signal retreat?''.

I guess the commander should act somewhat like ''ghosts'' act in certain games (in ''ghost mode''). They should be a reflection of all the actions the player has taken in the past and based on that should be able to somewhat act on its own to support the player.

Share this post


Link to post
Share on other sites
Ok, so your taking about "simulating the chain of command" so to speak, where *agents* below your command perform the micro-managment actions that you require. If you can come up with a design for this, it would really be cool. Computers are just so much faster then us at performing micro-tasks that games become a complete "click-fest" just to perform an attack or build up an army (no fun for me).

Do you have any such ideas as how to design this?

Maybe you can incorporate a rank for the player to move up, with AI NPC''s which are both below you and ABOVE, such as generals. Dont really know where that would go, but im sure you have thought of it.

Good luck.

Share this post


Link to post
Share on other sites
Why not ressurect an old post?

I noticed that you wanted one controlling AI instead of having Autonomous agents within each subgroup. I can see how that would drastically reduce system resources and would make "learning" a lot easier.

I think the only difference between our approaches is that in mine, there is an Agent in every single Organized Unit. On the scale that I''m thinking about, this is an issue, as I can envision having literally thousands of OU''s in play at a time, each one requiring a Commander. I could have the game go turn based, which would relieve a lot of computational strain, but I don''t think it would do justice to what I have in mind.

The reason I was thinking of having lot of little commanders rather than one big one was to watch how they all interplayed with one another as well as model how decision making capacity flows.

I was wondering if in your style, you would have customizable maneuvers/formations/tactics. WHen you design your military from the ground up, the other player will never know exactly what he is facing, and therefore the dilemma of RPS is abolished.

I also agree entirely on story. I want to have a backdrop story in a campaign mode. Indeed campaign mode will (or should be) the most fun to play. I''m not sure if I want ala StarCraft style or not though...I''m still trying to figure out how to integrate that.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
There is something to be said for being able to tweak parameters before a game is actually played.

For example, if people are familiar with Dark Reign, in that game you could set some different behaviors for units, like how aggressive they are, when they retreat, etc. Many units really benefitted from the proper settings. The problem was, you couldn''t do it before the game started, you had to do it when you first produced that type of unit. Which was very annoying because you had many other things to think about. And you ended up with the same settings every time anyway. There is no reason why you couldn''t define a set or multiple set of preferences before hand.

Anything that offloads that sort of stuff to the leisure time between games would be nice.

Share this post


Link to post
Share on other sites
quote:
Original post by Dauntless
I think the only difference between our approaches is that in mine, there is an Agent in every single Organized Unit. On the scale that I''m thinking about, this is an issue, as I can envision having literally thousands of OU''s in play at a time, each one requiring a Commander. I could have the game go turn based, which would relieve a lot of computational strain, but I don''t think it would do justice to what I have in mind.

The reason I was thinking of having lot of little commanders rather than one big one was to watch how they all interplayed with one another as well as model how decision making capacity flows.



I agree, it would be awesome if every unit had it''s own AI. You could then make clear distinctions between veteran and green troops: veteran troops have a more evolved AI, but may or may not have been trained in the manner you want. Furthermore, they learn slower. Green troops on the other hand, are fairly stupid, but they can be moulded to your wishes. Eventually, if your units are persistent, they will become valuable veteran troops, trained just how you want them.

An alternative, more feasible solution lies somewhere between the two extremes. One idea could be one or more ''Lieutenant'' units. Each Lieutenant unit has a learning AI, whereas ordinary units have only a basic, fixed AI. An OU with no lieutenant interprets your orders according to the fixed AI: an OU with a lieutenant carries out orders to your lieutenant''s interpretation of your orders: however he has been trained. At the most basic, the lieutenant simply parrots your orders, so there is no disadvantage to using green lieutenants, but as they learn they become a very useful resource.

One thing is clear, I need to do a lot more research on this subject before I decide exactly how it should work. Time to start hanging around the AI forum a bit more methinks.

quote:

I was wondering if in your style, you would have customizable maneuvers/formations/tactics. WHen you design your military from the ground up, the other player will never know exactly what he is facing, and therefore the dilemma of RPS is abolished.



This was pretty much the motivation for the concept. I envision training your units to use set pieces. Take Star Trek, when they say stuff like ''Attack Pattern Delta'' and the pilots then go off and execute a complex series of manouevres: I would like to be able to train my troops to use learnt attack formations at a single keypress.

quote:

I also agree entirely on story. I want to have a backdrop story in a campaign mode. Indeed campaign mode will (or should be) the most fun to play. I''m not sure if I want ala StarCraft style or not though...I''m still trying to figure out how to integrate that.


For the time being, I''ve decided to drop storyline almost completely. The main reason for this is that I am looking to produce a kind of proof of concept which demonstrates the gameplay principles in action, rather than a ''complete'' game at this stage. However, persistent AI units in a campaign game remain an interesting possibility. Maybe for the sequel....

Share this post


Link to post
Share on other sites
Seems like fun but i think there might be a few problems if not implemented correctly.

The map the game is played on has to be very random to avoid most effective build order scripts/ai''s which have proven to be the most effective in any situation.

Dont take to much control from the player to the point it basically becomes a game where the smartness of the commander decides the outcome. A press "W" to win sorta thing.

Share this post


Link to post
Share on other sites
quote:
Original post by DaJoostMan
Seems like fun but i think there might be a few problems if not implemented correctly.

The map the game is played on has to be very random to avoid most effective build order scripts/ai''s which have proven to be the most effective in any situation.

Dont take to much control from the player to the point it basically becomes a game where the smartness of the commander decides the outcome. A press "W" to win sorta thing.



I think that is very unlikely. The objective is not to teach the AI to win a war, but to teach it tactics which the player can use to win a war. And if an AI can exploit an unbeatable tactic than so can a human player, and if you have unbeatable tactics in your game then it is seriously flawed.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
The point of this would be to reduce the amount of clicking a player had to do to execute various tactics- not to make them a better player. Maybe if it was a scripting language you could be limited to 10 if..then..else statements or something. If everytime you retreat you select the group of units move them to the retreat point and leave X number of units to fight a rear-guard action that''s at least 5 or 6 clicks depending on how big the retreated group should be. With this you could just press R and then shift your attention.

Share this post


Link to post
Share on other sites