[.net] Threading: One producer - Many consumers : How to Synchronize ?
Author
Hi all,
I'm currently working on a piece of code written in C# where I need to synchronize one producer and many consumers. The producer is executed in one thread and each consumer runs in their own thread. The producer fills a queue with items (tasks) to be performed. When the queue has been filled the producer shall notify the consumers to start executig the enqueued tasks. Each consumer shall pick a task from the queue, run it and then pick the next task. If there are no more tasks the producer shall be notified. As long as there are tasks to run the producer shall simply wait.
I have implemented this for a simpler case where there is one producer and one consumer by using AutoResetEvents and it worked without any problems. However, I'm struggling with an algorithm for the case were there are multiple consumers. In particular with the case of notifying the producer when the Queue is empty (all tasks have been executed). To notify about when the queue is empty is one thing, but then how to notify the produer when the final task is completed? The queue could be empty but the last task could still be running.
So, if any of you have done something similar what strategy did you use?
Best Regards,
Mr_Bubbas
Unfortunately I'm not really familiar with C# past .Net 2.0. Java has a construct called a blocking queue that is perfect for the producer-consumer pattern, however I don't know if C# has an equivalent.
When googling though, I stumbled across this MSDN article which looks exactly like your question. ;)
When googling though, I stumbled across this MSDN article which looks exactly like your question. ;)
I suppose it depends somewhat on your scenario. In my case, targetting the 360 with XNA, I chose to attempt a lockless task pool for maximum possible utilization of all the cores.
The interface is pretty standard, with the task component exposing a DoTasks method that takes a list of tasks, and blocks until all tasks in the queue have been completed. Internally it uses a shared "next task" index consumed by all the threads, and incremented safely with Interlocked.CompareExchange. I used ManualResetEvents in an airlock pattern to stop the worker threads once all work items have been consumed (A closes, B opens, threads run tasks then block on A. B closes, A opens, threads run tasks then block on B).
I get about as close to 100% cpu utilization as I think I can reasonably expect to get for my app, so I wrote up a journal post about it and included the source. If you're interested, you can find it here. I hope it helps.
The interface is pretty standard, with the task component exposing a DoTasks method that takes a list of tasks, and blocks until all tasks in the queue have been completed. Internally it uses a shared "next task" index consumed by all the threads, and incremented safely with Interlocked.CompareExchange. I used ManualResetEvents in an airlock pattern to stop the worker threads once all work items have been consumed (A closes, B opens, threads run tasks then block on A. B closes, A opens, threads run tasks then block on B).
I get about as close to 100% cpu utilization as I think I can reasonably expect to get for my app, so I wrote up a journal post about it and included the source. If you're interested, you can find it here. I hope it helps.
Author
Rycross -> Thanks for the link. I somehow managed to miss that MSDN article.
JPatrick -> I'm also targetting the XBox360 and from a quick look at your code it looks like you've done something quite similar to what I'm trying to do. Looks like a very interesting read. I'll read it this evening.
Thanks for your help guys. It really helped :)
Best Regards,
MR_Bubbas
JPatrick -> I'm also targetting the XBox360 and from a quick look at your code it looks like you've done something quite similar to what I'm trying to do. Looks like a very interesting read. I'll read it this evening.
Thanks for your help guys. It really helped :)
Best Regards,
MR_Bubbas
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
