But I think, that BHT and MCTS are not really comparable. BHTs are more or less advanced version of a scripted AI. So, regardless of how good your MCTS works, I believe that you could create a BHT which will top it. The motivation to use a the BHT in game development is to be able to design the behaviour (a reason these are so much loved in game development), so arguable not really AI at all to be honest,
Thanks for the feedback. :) The thing is, today almost all behaviour is created using BT (or some technique similar, like FSM), but the more (machine learning) modern techniques are rarely used, maybe due to their unpredictable nature and being computational heavy. But the progression of computer power allows for more advanced techniques to be used while also, undeniable, sets the expectations for AI a lot higher. This means that hand-coding AI becomes too complex, which requires techniques (such as MCTS) to step in and help. Another thing is, once you figure out how the scripted AI works you can beat it every time. This could potentially be avoided using more modern techniques.
What I compare in my study is how both techniques affect user enjoyment, which I find to be a fair comparison. Both techniques are to be used for the same purpose, control the AI, while behaving differently.
As for whether or not a BT can top MCTS is of course hard to figure out, however, I have found a study showing how a rule-based AI, developed over a period of 10 years, being beaten by a simple MCTS implementation (Integrating Monte Carlo Tree Search with Knowledge-Based Methods to Create Engaging Play in a Commercial Mobile Game). I feel this shows the strength of such a technique.
Another experience I had when implementing the two techniques was the extreme flexibility of adding new units for the MCTS to handle. I just had to specify what the new unit was allowed to do, and the MCTS would take of the rest.
