Sign in to follow this  
jolyqr

Reinforcement Learning

Recommended Posts

Do someone knows what reinforcement learning, Q-Learning, Markov Decision Process and SARSA are about? Basically I'm doing some researches on ANN to control a NPC AI. I could use a backpropagation net to make my NPC learn, but this technique is not aimed to be used in real time processes. I'm looking for a way to allow the NPC learn in real time. I have read some articles dealing with "Reinforcement Learning". I think I could use the reinforcement learning with ANN. cheers!! [Edited by - jolyqr on June 8, 2006 12:52:23 PM]

Share this post


Link to post
Share on other sites
Quote:
Original post by jolyqr
Basically I'm doing some researches on ANN to control a NPC AI.

What attributes in the NPCs state space are you trying to control? What sort of control problem are you wanting to solve (regulation, compensation, etc.)? Without first specifying the control problem, its very hard to talk about learning a control function (so called "intelligent control").

Quote:

I'm looking for a way to allow the NPC learn in real time.


This is a significant and difficult open research problem. Online learning of control functions is extremely problematic, even more so when you don't have a supervisory control system to learn from.


Quote:

I have read some articles dealing with "Reinforcement Learning".
I think I could use the reinforcement learning with ANN.


If you want to learn the parameters of an ANN while using it for a given task, then you want to solve a dual estimation problem. Off the top of my head, I think Eric Wan did some research in this area. If that's a dead end, let me know and I'll dig through my bibliography file for a specific reference.

Cheers,

Timkin

Share this post


Link to post
Share on other sites
Theoretically, the problem I want to solve is not too complicated. I would like my NPC to fire the player, to look for items and to flee…

I really don’t know a lot about reinforcement learning, so I don’t know if I need to do some “regulations” or “compensations”…

I know this is a very difficult area. My supervisor told me it’s possible to do some reinforcement learning in ANN. I have taken a look on Internet, I did not find a lot on using a ANN in reinforcement learning. The only examples I found use q-learning…

Anyway, if you have any information concerning that matter, don’t hesitate to give it to me…

cheers

Share this post


Link to post
Share on other sites
Quote:
Original post by jolyqr
Theoretically, the problem I want to solve is not too complicated. I would like my NPC to fire the player, to look for items and to flee…


The typical approach to this sort of problem is to break down the agent's behaviours into those that can be handled by a single AI methodology and then to combine them into your final agent. For example, your agent might make use of a pathfinder for deciding how to get to spatial goals, a planner for choosing those goals, a reactionary system for moving in combat, a decision system for choosing targets in combat, a sensory system for determining the world context, etc, etc. One or more of these systems may be in operation at any time and they will be changing the state of the agent accordingly.

If you want to investigate the use of reinforcement learning for your game agent, I would suggest that you choose one problem (such as target selection) and concentrate on that. To attempt to create a complete agent that learns all of its behaviours as mimicry of a human expert would be an exceedingly large task (and indeed, would probably be achieved by combining several smaller learning systems).

Cheers,

Timkin

Share this post


Link to post
Share on other sites
Quote:
Original post by Timkin
Quote:
Original post by jolyqr
Theoretically, the problem I want to solve is not too complicated. I would like my NPC to fire the player, to look for items and to flee…


The typical approach to this sort of problem is to break down the agent's behaviours into those that can be handled by a single AI methodology and then to combine them into your final agent. For example, your agent might make use of a pathfinder for deciding how to get to spatial goals, a planner for choosing those goals, a reactionary system for moving in combat, a decision system for choosing targets in combat, a sensory system for determining the world context, etc, etc. One or more of these systems may be in operation at any time and they will be changing the state of the agent accordingly.

If you want to investigate the use of reinforcement learning for your game agent, I would suggest that you choose one problem (such as target selection) and concentrate on that. To attempt to create a complete agent that learns all of its behaviours as mimicry of a human expert would be an exceedingly large task (and indeed, would probably be achieved by combining several smaller learning systems).

Cheers,

Timkin


ok. cheers

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this