Sign in to follow this  

How to Design for Threading

This topic is 3313 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm not sure how this works so I'll paint a hypothetical picture to stand-in as a good example... Say I am designing a web app that will contact a server-side DLL (C# .NET assembly) when a user does something (e.g., clicks a button). The client sends some form of an XML message that the serverside DLL - aka "Widget.dll" - reads in as data and processes. The problem here is that there could be tens of thousands of simultaneous users at any given time. I am looking for elegant solutions that (will most likely involve multi-threading and) guarantee my Widget.dll doesn't get bogged down with all the traffic and trying to process all these XML requests. I have become familiar with System.Threading and have some questions: (1) Is there an upper-limit to the number of threads one can/should open for a specific instance of an object? I can't seem to find any docs/ref material on this? (2) Are there any algorithms for calculating the number of threads that should be opened at any given time, or more generally, any algs that would be applicable and useful to this problem (from a resource mgmt perspective)? (3) Are there other elegant ways of acquiescing massively-multiple users? For instance, perhaps having multiple servers, and/or multiple Widget.dll's running on each server? thx, ply

Share this post


Link to post
Share on other sites
Ideally, the number of threads will be closer to the number of cores than to the number of users/connections/whatever. For instance, you can set up a listening thread that receives incoming requests and pushes them onto a queue, then use one worker thread per core to extract requests from the queue and serve them. This is because if you have four cores, you can only serve four requests at a time, and any attempt to serve more will just slow everything down.

The only special case is when your worker threads have to wait on an asynchronous event (such as querying another server). In that case, you can have several threads per core in order to make use of the free time while other threads are waiting.

Share this post


Link to post
Share on other sites
The general rules are:

Keep the number of threads to a minimum, as it takes significant time to swap threads.

Keep the number of CPU bound threads equal to the number of logical threads your processor supports.

For MEMORY bound threads it can be tricky, since logical threads your processor supports can be higher than the number of cores / cache banks. This means you want AT MOST a number of threads equal to the number of cores for some setups to avoid cache thrashing.

You can have many IO bound threads, but keep the number as low as possible.

What you probably want is a thread pool of CPU bound workers to process your requests, and then one thread to recieve the requests and enqueue the data for the processing pool.


--addendum.
A dual core xeon with hyperthreading is 1 processor die, 2 processor cores, and 4 logical threads.
Some spark servers support hundreds of logical threads per core. Other processors only support one logical thread per core.

Share this post


Link to post
Share on other sites
It seems you're using some web app framework, and you merely provide a plug-in dll.

In that case, threading will be performed by server as needed.

Even without it, there's little need for threading for purposes of scalability. Only exception would be blocking calls in third-party API (SQL for example), in which case running those in separate thread pool of workers may help. Choosing a non-blocking API however is likely to be considerably more scalable.

Main reason for this is that this type of plugins is usually request centric, giving you little room for global management of resources. This simply means you'd need to create/destroy threads for each request you receive which is a disaster.

Share this post


Link to post
Share on other sites

This topic is 3313 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this