• Advertisement
Sign in to follow this  

AI User study :)

This topic is 972 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi all,

currently I'm in the process of making a user study where I try out different AI techniques and see what the effect might be on players. For the experiment I've implemented two types of AI: one using behaviour trees; and one using Monte Carlo Tree Search. The techniques are used in a turn-based tactic game. 
I would really love some feedback on what you think. The game has some questionnaires between each AI opponent, and it would amazing if you could play the game 4+ times (a new game will come after each questionnaire). The game can be found here:

http://www.janolesen.org/game.html (I think my DNS is down, alternatively use link below)

http://139.162.148.166/game.html

 

As the game is made in Unity3D Chrome is unfortunately not an option at the moment. So I suggest you use Safari, Firefox, or (dare I say) IE.

Thanks in advance!

 

ingame.png

ingame.png

Edited by sqdejan

Share this post


Link to post
Share on other sites
Advertisement


one using behaviour trees; and one using Monte Carlo Tree Search. The techniques are used in a turn-based tactic game.
I would really love some feedback on what you think.

Great idea smile.png

 

But I think, that BHT and MCTS are not really comparable. BHTs are more or less advanced version of a scripted AI. So, regardless  of how good your MCTS works, I believe that you could create a BHT which will top it. The motivation to use a the BHT in game development is to be able to design the behaviour (a reason these are so much loved in game development), so arguable not really AI at all to be honest,

Share this post


Link to post
Share on other sites


As the game is made in Unity3D Chrome is unfortunately not an option at the moment. So I suggest you use Safari, Firefox, or (dare I say) IE.

 

You can use Unity 5's WebGL exporter option for Chrome.

Currently in the process of porting one of my games, and save for a few graphical issues, it works like a charm.

Just sayin :)

Share this post


Link to post
Share on other sites

Is the first enemy random? My reasoning is that your results will be incorrect if this is a fixed order as players will learn the game in the first round (as I did) and are prone to behave poorly against the first enemy (as I did).

The issue is that I didn't quite know how to move AND attack on the first turn until halfway through the first match (which I've lost) but afterwards, I haven't lost a single match.

Share this post


Link to post
Share on other sites


But I think, that BHT and MCTS are not really comparable. BHTs are more or less advanced version of a scripted AI. So, regardless  of how good your MCTS works, I believe that you could create a BHT which will top it. The motivation to use a the BHT in game development is to be able to design the behaviour (a reason these are so much loved in game development), so arguable not really AI at all to be honest,

 

Thanks for the feedback. :) The thing is, today almost all behaviour is created using BT (or some technique similar, like FSM), but the more (machine learning) modern techniques are rarely used, maybe due to their unpredictable nature and being computational heavy. But the progression of computer power allows for more advanced techniques to be used while also, undeniable, sets the expectations for AI a lot higher. This means that hand-coding AI becomes too complex, which requires techniques (such as MCTS) to step in and help. Another thing is, once you figure out how the scripted AI works you can beat it every time. This could potentially be avoided using more modern techniques.

 

What I compare in my study is how both techniques affect user enjoyment, which I find to be a fair comparison. Both techniques are to be used for the same purpose, control the AI, while behaving differently.

 

As for whether or not a BT can top MCTS is of course hard to figure out, however, I have found a study showing how a rule-based AI, developed over a period of 10 years, being beaten by a simple MCTS implementation (Integrating Monte Carlo Tree Search with Knowledge-Based Methods to Create Engaging Play in a Commercial Mobile Game). I feel this shows the strength of such a technique. 

 

Another experience I had when implementing the two techniques was the extreme flexibility of adding new units for the MCTS to handle. I just had to specify what the new unit was allowed to do, and the MCTS would take of the rest.

Share this post


Link to post
Share on other sites


You can use Unity 5's WebGL exporter option for Chrome.
Currently in the process of porting one of my games, and save for a few graphical issues, it works like a charm.
Just sayin

 

Thanks!!! Will look into that.

 


Is the first enemy random? My reasoning is that your results will be incorrect if this is a fixed order as players will learn the game in the first round (as I did) and are prone to behave poorly against the first enemy (as I did).

 

It's not random (I think you played against MCTS?). It is just more reactive to how you play, trying to figure what is the best move against the current state of the board. I've been inspired by the fixed order from Heroes of Might and Magic, because I thought an AI like MCTS would work perfect in a scenario like that.

 


The issue is that I didn't quite know how to move AND attack on the first turn until halfway through the first match (which I've lost) but afterwards, I haven't lost a single match.

 

Good point, will look into that.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement