Jump to content
  • Advertisement

Archived

This topic is now archived and is closed to further replies.

daerid

Most Efficient High-Performance I/O model?

This topic is 5862 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I''m developing a custom SMTP server application that''s expected to be receiving anywhere from 3000 to 5000 connections/emails per hour. My current implementation creates a new thread to handle each client connection, and utilizes overlapped I/O. What I''m wondering is if using I/O Completion ports would be a better idea. What are the pro''s and con''s to each approach?

Share this post


Link to post
Share on other sites
Advertisement
IO Completion ports are the Microsoft preferred method of handling Winsock IO for large-scale servers. The model fits well with a thread-pool and performance is the best you''ll get if it''s written well. But like any piece of code that you need to see good performance from... profile, profile, profile.

Share this post


Link to post
Share on other sites
One disadvantage to using I/O completion ports (which I''m ALL for for) is that they are only available on WinNT/2000/XP. Still, one thread per client is not a scalable solution under Windows. While you could create 3000 threads and service clients that way, the number of context switches and cache misses would be brutal and kill the performance of your server. I/O completion ports, under NT/2000/XP, are definitely the way to go. I/O completion ports are stupid-easy (technical term ) to use under Windows 2000/XP because of the thread-pool management functions (which use I/O completion ports internally.)

Dire Wolf
www.digitalfiends.com

Share this post


Link to post
Share on other sites
Thread pool management functions?


News to me. Could you elaborate on this a little more? I''d really appreciate it

Share this post


Link to post
Share on other sites
One thing I noticed about BindIoCompletionCallback() is that the callback function is called by a non I/O thread. Perhaps I''m wrong, but doesn''t this mean you can''t post more I/O requests inside your callback? Most servers will probably get their first completion packet as a result of a call to WSARecv(), then want to post another I/O request with WSASend().

Would it be better to instead call QueueUserWorkItem() with the WT_EXECUTEINIOTHREAD flag, and then call GetOverlappedResult() in your callback to get the result of the I/O?

Matt

Share this post


Link to post
Share on other sites
Well, the key to squeezing out all the performance you can is keeping the IO Subsystem busy. If you queued things to different threads using APC, that means you''d be waiting for the scheduler to make a context switch to the right thread to initiate the send. With IO Completion ports, generally I''d think that you would send from any thread you want, and just let the IO threads handle your IO Completion for sends.

Receiving would be slightly different, it has to be more planned, since when you call WSARecv, you''re really just supplying a buffer to be used when it''s needed most of the time, but there could also be waiting data so your WSARecv might complete right away. You''d probably want to figure out some ideal number of receive buffers you''d like posted when you create the socket, allocate them and call WSARecv on them, then just rotate those buffers through calls to WSARecv. That way you don''t have memory allocation / thrashing issues for your buffers.

Share this post


Link to post
Share on other sites
What I''d like to know is whether it''s ok to post additional overlapped I/O requests from inside your callback. Jeffrey Richter wrote in an article on MSDN that you can''t, because the non-IO thread which calls your BindIoCompletionCallback routine could be terminated with your outstanding I/O still pending. The platform SDK contradicts this, stating that both I/O and non-I/O threads can initiate I/O requests, and that neither would be terminated with I/O still pending. Any clarification on the matter?

Matt

Share this post


Link to post
Share on other sites
Read it more carefully:

quote:

From Platform SDK, August 2001 Edition
A non-I/O worker thread waits on I/O completion ports. Using non-I/O worker threads is more efficient than using I/O worker threads. Therefore, you should use non-I/O worker threads whenever possible. Both I/O and non-I/O worker threads do not exit if there are pending asynchronous I/O requests. Both types of threads can be used by work items that initiate asynchronous I/O completion requests. However, avoid posting asynchronous I/O completion requests in non-I/O worker threads if they could take a long time to complete.



This description really isn''t that clear but it essentially says, "If you post an overlapped I/O request to a non-I/O work thread, if the operation doesn''t complete in a timely manner, the thread might terminate and all pending overlapped operations will be cancelled." That is my interpretation of the information with a bit of help from Jeffrey Richter

The idea is that you receive I/O completion notifications on non-I/O threads but you queue I/O requests to I/O threads using QueueUserWorkItem(). When the work item is processed in the I/O thread, an overlapped I/O request is made against the socket/file handle. The call to the asynchronous function returns immediately and allows the I/O thread to process more work items. Since you bound the socket handle to the thread pool, when the overlapped I/O operation completes a callback is made on a non-I/O thread. Then the process repeats.

Hope this clears things up.

Regards,





Dire Wolf
www.digitalfiends.com

Share this post


Link to post
Share on other sites
3000-5000 connections per hour is a miniscule amount of connections. For this appliation it doesn''t matter how you handle connections...(assuming they won''t all be active at once). Even creating a new thread for each connection will work in this case. Just code it in perl for that matter. Bothering with I/O completion ports for this is a waste of time unless your traffic is going to grow a hundredfold.

Share this post


Link to post
Share on other sites

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!