Hiya,

Sorry for all the ASIO related threads recently. Thanks for everyones help, I'm almost there :)

My application listens for incoming connections and then runs through an authorisation protocol with the clients. I'm using the boost ASIO and thread libraries.

At present, I'm using one boost::io_service and several threads to handle the connections. So different read/write operations from the same client could be handled by different threads.

The ASIO callbacks need to access a database to authorise clients, so each worker thread will need it's own database object (this is how the DB library is designed - without concurrency). The callback each thread is executing should then use that threads' db object. I'm not sure how to go about this though, as the callbacks for a given socket can be executed by any worker thread... How can I tell a callback which db object it should use?

Could I use thread-local storage for this? And access it from within the callbacks? Or someone suggested previously I could use boost::bind somehow..

Any help/advice/opinion is really appreciated, thanks! :)

You almost certainly don't need this level of complexity at this point in time. When you do, you'll have a much better idea of how to code it.

Have your boost::asio tasks read messages off the sockets. Fixed length ones are nice & easy to handle. When the callback returns, it appends the message to a queue and restarts the read.

Writing works similarly -- when the callback happens pop the message off the queue and start another write for the next one.

You'll need a mutex on the message queues and a master semaphore for the main loop which is signalled when data arrives from a client. The main loop simply does "sleep on semaphore. Process all incoming messages on all queues. Loop".

You'll need some simply "triggers" for the case where there's no data to send, so you didn't start a write, but another message has just been enqueued; a gotcha here is that if you use flags to say whether there's a write in progress or not, you'll need to make sure that you protect the flags with the mutex.

The main loop holds the DB connection, and since access to that is single threaded, there's no contention problem.

If you're struggling with DB operations taking too long, then a pool of worker threads who are handed work by the main loop and send messages back to be returned to clients is an easy extension.

Each threaded system is then doing one thing and has one resource to worry about, and that makes them a lot easier to code.

Quote:Original post by beebs1
Could I use thread-local storage for this? And access it from within the callbacks? Or someone suggested previously I could use boost::bind somehow.. help/advice/opinion is really appreciated, thanks! :)

I think that TLS works fine for this task and I believe it is the easiest solution that would not require locking or some other sort of synchronized look up access if you go that direction. In my prototype solution I wrote up to test out what you were asking in your other thread, I used TLS and it seemed to do exactly what you need. I only tested with SQLite3 though which requires a locking mutex anyway since I don't have anything else handy on this machine, but the concepts seem to be sound through debugging. As long as your connector supports handling that kind of multiple connections to the database fine, you should be 'ok'. Up until thinking about a solution for your problem if I were tasked with solving it, I'd not really looked into using TLS, but referring to MSDN gives you all of what you need to get started as usual with most of the Win32 API functions.

There is another problem you have to address though that can be easily overlooked. Assuming you keep everything independent of each other thread wise, how are you going to manage zombie connections with no list to reference? Consider if someone connected to your server and began the authentication process but never finished. You can't use a thread specific list because a different owning thread might process the event, so you'd have to post it to that threads event queue; which I'm not sure is even possible with boost::thread. You can't cheat and use a locked list of some sort because there goes your whole solution of everything being independent. Some sort of lock free structure is out of the question too due to unnecessary complexity it adds. I won't post exactly how I solved that issue, even though it is literally only a handful lines of code. I will just say though boost gives you -everything- you need to solve the problem and you will come to understand how useful boost::bind really is!

Aside from that, I've not really come across any issues that would prevent your original proposal from working (assuming the whole thread connection to database concept works flawlessly). Yes, there are other methods that might be simpler and less complex as was mentioned in the other thread and what Katie mentioned here, but I certainly think your proposal is do-able and I don't see anything overly complex or hard about it. I think you have a well defined problem and the implementation for it is not that challenging assuming you have the boost knowledge already. I mean after adding some pending write handling logic (required so you don't have two different threads trying to post writes on the same socket) then adding protocol specific handling to interface to your other logic for auth, you should be easily around only 350 lines or so of core code including white space. My prototype which hadn't implemented write logic is at 285 right now and the code is overall none too clean so that's how I got that estimate. I have the write logic done as per another project, just didn't copy it over yet.

The last nagging issue you might come across is how do you easily exit the server without force closing it if all your threads are waiting for one another to run out of work. Here's a tip on that, calling io_service.stop function stops all work, so if you setup another thread that just blocked waiting for the exit event request, you can then call the function and the system cleans up nicely!

I think that's about all there is to it. It is somewhat on the more complex side for starting out but you never know until you try. The majority of the learning curve here comes from boost. If going this route leads you to headaches and programming pains, then you can do something more simple and then come back later to work out what what wrong as Katie mentioned. It's easy to fall into the trap of thinking you have to make the absolute best solution on paper with this networking stuff, but the truth of the matter is until you have a working baseline to compare to, starting really simple can usually yield surprising results as being a "good enough" solution that works and is something you can easily maintain and understand. Happy coding!