Hello,
I am learning MT (using boost.threads) right now. And I have been google-ing all day today about Multithreading, its issues and how to prevent them. But, I still don't understand one thing about communication between threads. I am trying to make an event handler to communicate between threads but I am getting confused everytime i start to think about it. Here is my scenario:

I have 3 threads for now: Core, Graphics, and Scripting
Each thread has a game system. Game systems have functions to handle the events sent by the components. The events are always read only (they are ONLY created by the Component).

Core Thread:
It runs Logger, Filesystem, Input, Game Entities (should I change it to a new thread?)

Graphics Game System (also thread):
Contains all the instances of models, animations etc (using OSG) and renders every graphics related object

Scripting Game System (also thread):
Contains and Runs Python Scripts.

Everything works asynchronously. But here is whats confusing me and not letting me move forward.

I have a HealthComponent and for testing purposes I put:


if(parent->attribute("health") < 50) {
GameSystemHandler::passEvent(new Event1<String>("ChangeAnimation", "AnimationCrippling")); //it sends events to all the game systems (is in the core thread).
}


Now, if two entities got shot hard that they lost 70 health out of 100, How GraphicsSystem is going to take care of changing both entities' animations. Isn't it going to overlap??
And the same for Scripting. If two of them says to run some script that has:


engine.current_object.doSomething();


How does the threading know which current_object it is?

In both of these casec, do I have to use mutexes to stop one until the other one finishes? If I do, wouldn't that be a HUGE bottleneck and MT is going to be the same performance of a single threaded environment (concurrency is going to get destroyed due to the syncs).

I am really new to the MT theory. Please help me out!

Thanks in advance,
Gasim

I am really new to the MT theory. Please help me out!

This, I believe, is the source of the problem.

I suggest you STOP implementing your multithreaded code in your engine, and instead learn how it works.

You will start with the equivalent of writing multithreaded "Hello, World!" type applications. Keep practicing until you are comfortable writing multithreaded code that handles producer/consumer models, and can comfortably write code that communicates common objects without requiring locks.


Debugging badly-written parallel code is incredibly more difficult than debugging badly-written serial code. Don't add the complexity if you don't need to.


After you have become competent with it, and after you've determined that the extra complexity and the hair-pulling bugs are worth it, that's when you should begin adding it to your game.

I have 3 threads for now: Core, Graphics, and Scripting
Core Thread:
It runs Logger, Filesystem, Input, Game Entities (should I change it to a new thread?)
Graphics Game System (also thread):
Contains all the instances of models, animations etc (using OSG) and renders every graphics related object
Scripting Game System (also thread):
Contains and Runs Python Scripts.
[/quote]

That's an odd way to partition your work.

Any time you work with parallel processing consider something covered in most good multiprocessing text books:

PCAM.

Partition - break the work into the smallest possible part of work. Every task has a serial portion that cannot be broken down further. Try to reach that.
Communication - Figure out communication patterns between those parts.
Agglomeration - Group together things that communicate closely or are interrelated.
Mapping - Now that you understand the flow, map the chunks of work into their separate processes.


As you correctly pointed out, with your design even simple events will require crossing thread boundaries many times. This is a poor design. It means you have sliced your work right down the middle of a communications pattern, rather than keeping communications and interactions to a minimum. In addition to more headaches, the interactions between processes requires extra synchronization and blocking requirements that can easily make your code run slower than the single-thread version.



Go back to the beginning, learn multiprocessing outside your game engine. Only after you have a solid understanding should you start introducing the complexity and bugs that come with it into your code.

Oh. I now read it again and found out that my design was flawn. Now i changed my draft.

[color=#1C2837][size=2]The events will need to include what object the event occurred on, or some identifier that allows the recipient to handle the event properly.[/quote]

Handlers take care of that:

typedef std::map<String, Handler> HandlerMap


where String is the event's name.

[color=#1C2837][size=2]And yes, doing a whole lot of synchronization between threads destroys performance, and yes, designing a system to avoid that effectively is hard.[/quote]

Do you know any good open source game engines with effective MT support. I just want to read the code and see how they implemented it. Or is there any good resource (book or article) where its covered pretty well.

Thanks in advance,
Gasim

[color=#1C2837][size=2]

[color="#1C2837"]And yes, doing a whole lot of synchronization between threads destroys performance, and yes, designing a system to avoid that effectively is hard.

Do you know any good open source game engines with effective MT support. I just want to read the code and see how they implemented it. Or is there any good resource (book or article) where its covered pretty well.
[/quote]
Perhaps the word "hard" is too vague. Let's give some more concrete examples.



One good resource is getting parallel processing as an emphasis in your college degree. It takes a collection of college courses, so it isn't a simple book or article. That would be enough to get you started. If you aren't in school you're looking at 3-6 months of focused study and multiple sample projects, along with multiple books and online resources. That is enough to cover the basics. That would be basic entry-level fluency for work in multiprocessing. It is not enough to design large systems, but enough to understand the designs developed by very experienced senior engineers.

If you are not in school, an alternate resource is to work in the trenches with multiprocessing code for 3-5 years, make a bunch of mistakes, watch as your co-workers make similar mistakes, and get the knowledge through on the job training. Again, that will give you basic fluency to work with it, but probably not enough to design it from scratch.

This is not an easy subject. It can require YEARS of study and practical experience to simply reach basic competency at designing and building multiprocessing systems.

It is not something you will pick up with a few online tutorials and studying how a game implemented it.




Consider a recent series of articles about how to design a lockless queue. It appeared in DDJ, a very old and established programming magazine. A lockless queue is a fairly simple and straightforward data structure that is frequently used in multiprocessing.

First there was the original article about it accompanied by source code. It was written by an experienced developer who knew quite a lot about parallel processing, and it was peer reviewed for correctness. In spite of that, it had several subtle bugs that could be exposed in race conditions. They issued a followup article, explaining just how even a relatively simple parallel concept of a lockless queue had such subtle issues that needed to be address. They carefully explained what they got wrong for the bugs they had found, and corrected the system. That article in turn was peer reviewed by more experts before publishing.

Then the following month, one of the most respected parallel processing experts in the world wrote a THIRD article for the magazine, explaining how both the original and follow up article made some systemic errors that are fairly common even for experts to make, and explained exactly how they needed to be corrected, along with further admonition that properly designing and developing parallel code is hard, even for those with many years of real world experience at it.


Parallel processing and the "For Beginners" forum are generally a bad combination. It is an excellent way to lose your sanity to some amazingly complex and subtle bugs.

[color=#1C2837][size=2]... which has no reference to the actual object where the event is taking place. If two entities are hit for 50, which one do you change?[/quote]

[color=#1C2837][size=2]

[color="#1c2837"]I have figured that one out. Since, game entities are all stored in one single thread, sending events will be sent in sequential order to the other.
[color="#1c2837"]

[color="#1c2837"]I have found lots of articles. Currently reading and implementing them.
[color="#1c2837"]

[color="#1c2837"]Thank you everyone for help!

I haven't implemented it yet. its in my draft. Currently I am reading a lot of articles about all kinds of MT and implementing them and solving problems. So, I am going to start continuing my game engine after understanding MT really good.

Depending on your event system there is no guarantee one event will be handled before the other. It's good to treat the process of a context switch as the internet in terms of when things reach your server. Generally speaking, you have no control what gets where and when. Neither is there any control over if you receive both events on the other thread at the same time. Is it unlikely? Yes but if you run the code enough times I'm pretty sure you will see it happen. What's the fix to this? I'm unsure. You could implement a globally unique event id scheme whereby something akin to Guid.NewGuid() in .Net is used to send off events. Then you could evaluate these identifiers and choose either the greater or the lesser as which one to act upon first. However, given that specific implementation Guids are random so there is no control over which will be greater than the other. I'm not advising this approach. It's simply the only one I could come up with off the top of my head.