Jump to content
  • Advertisement
Sign in to follow this  
jolyqr

Reinforcement Learning

This topic is 4388 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Do someone knows what reinforcement learning, Q-Learning, Markov Decision Process and SARSA are about? Basically I'm doing some researches on ANN to control a NPC AI. I could use a backpropagation net to make my NPC learn, but this technique is not aimed to be used in real time processes. I'm looking for a way to allow the NPC learn in real time. I have read some articles dealing with "Reinforcement Learning". I think I could use the reinforcement learning with ANN. cheers!! [Edited by - jolyqr on June 8, 2006 12:52:23 PM]

Share this post


Link to post
Share on other sites
Advertisement
Quote:
Original post by jolyqr
Basically I'm doing some researches on ANN to control a NPC AI.

What attributes in the NPCs state space are you trying to control? What sort of control problem are you wanting to solve (regulation, compensation, etc.)? Without first specifying the control problem, its very hard to talk about learning a control function (so called "intelligent control").

Quote:

I'm looking for a way to allow the NPC learn in real time.


This is a significant and difficult open research problem. Online learning of control functions is extremely problematic, even more so when you don't have a supervisory control system to learn from.


Quote:

I have read some articles dealing with "Reinforcement Learning".
I think I could use the reinforcement learning with ANN.


If you want to learn the parameters of an ANN while using it for a given task, then you want to solve a dual estimation problem. Off the top of my head, I think Eric Wan did some research in this area. If that's a dead end, let me know and I'll dig through my bibliography file for a specific reference.

Cheers,

Timkin

Share this post


Link to post
Share on other sites
Theoretically, the problem I want to solve is not too complicated. I would like my NPC to fire the player, to look for items and to flee…

I really don’t know a lot about reinforcement learning, so I don’t know if I need to do some “regulations” or “compensations”…

I know this is a very difficult area. My supervisor told me it’s possible to do some reinforcement learning in ANN. I have taken a look on Internet, I did not find a lot on using a ANN in reinforcement learning. The only examples I found use q-learning…

Anyway, if you have any information concerning that matter, don’t hesitate to give it to me…

cheers

Share this post


Link to post
Share on other sites
Reinforcement Learning: An Introduction
Richard S. Sutton and Andrew G. Barto

online version :
http://www.cs.ualberta.ca/~sutton/book/ebook/the-book.html

Regards
Michael

Share this post


Link to post
Share on other sites
Quote:
Original post by jolyqr
Theoretically, the problem I want to solve is not too complicated. I would like my NPC to fire the player, to look for items and to flee…


The typical approach to this sort of problem is to break down the agent's behaviours into those that can be handled by a single AI methodology and then to combine them into your final agent. For example, your agent might make use of a pathfinder for deciding how to get to spatial goals, a planner for choosing those goals, a reactionary system for moving in combat, a decision system for choosing targets in combat, a sensory system for determining the world context, etc, etc. One or more of these systems may be in operation at any time and they will be changing the state of the agent accordingly.

If you want to investigate the use of reinforcement learning for your game agent, I would suggest that you choose one problem (such as target selection) and concentrate on that. To attempt to create a complete agent that learns all of its behaviours as mimicry of a human expert would be an exceedingly large task (and indeed, would probably be achieved by combining several smaller learning systems).

Cheers,

Timkin

Share this post


Link to post
Share on other sites
Quote:
Original post by Timkin
Quote:
Original post by jolyqr
Theoretically, the problem I want to solve is not too complicated. I would like my NPC to fire the player, to look for items and to flee…


The typical approach to this sort of problem is to break down the agent's behaviours into those that can be handled by a single AI methodology and then to combine them into your final agent. For example, your agent might make use of a pathfinder for deciding how to get to spatial goals, a planner for choosing those goals, a reactionary system for moving in combat, a decision system for choosing targets in combat, a sensory system for determining the world context, etc, etc. One or more of these systems may be in operation at any time and they will be changing the state of the agent accordingly.

If you want to investigate the use of reinforcement learning for your game agent, I would suggest that you choose one problem (such as target selection) and concentrate on that. To attempt to create a complete agent that learns all of its behaviours as mimicry of a human expert would be an exceedingly large task (and indeed, would probably be achieved by combining several smaller learning systems).

Cheers,

Timkin


ok. cheers

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!