Most Efficient High-Performance I/O model?
Author
I''m developing a custom SMTP server application that''s expected to be receiving anywhere from 3000 to 5000 connections/emails per hour.
My current implementation creates a new thread to handle each client connection, and utilizes overlapped I/O. What I''m wondering is if using I/O Completion ports would be a better idea.
What are the pro''s and con''s to each approach?
IO Completion ports are the Microsoft preferred method of handling Winsock IO for large-scale servers. The model fits well with a thread-pool and performance is the best you''ll get if it''s written well. But like any piece of code that you need to see good performance from... profile, profile, profile.
One disadvantage to using I/O completion ports (which I''m ALL for for) is that they are only available on WinNT/2000/XP. Still, one thread per client is not a scalable solution under Windows. While you could create 3000 threads and service clients that way, the number of context switches and cache misses would be brutal and kill the performance of your server. I/O completion ports, under NT/2000/XP, are definitely the way to go. I/O completion ports are stupid-easy (technical term 
) to use under Windows 2000/XP because of the thread-pool management functions (which use I/O completion ports internally.)
Dire Wolf
www.digitalfiends.com
) to use under Windows 2000/XP because of the thread-pool management functions (which use I/O completion ports internally.) Dire Wolf
www.digitalfiends.com
Author
Thread pool management functions?
News to me. Could you elaborate on this a little more? I''d really appreciate it
News to me. Could you elaborate on this a little more? I''d really appreciate it
One thing I noticed about BindIoCompletionCallback() is that the callback function is called by a non I/O thread. Perhaps I''m wrong, but doesn''t this mean you can''t post more I/O requests inside your callback? Most servers will probably get their first completion packet as a result of a call to WSARecv(), then want to post another I/O request with WSASend().
Would it be better to instead call QueueUserWorkItem() with the WT_EXECUTEINIOTHREAD flag, and then call GetOverlappedResult() in your callback to get the result of the I/O?
Matt
Would it be better to instead call QueueUserWorkItem() with the WT_EXECUTEINIOTHREAD flag, and then call GetOverlappedResult() in your callback to get the result of the I/O?
Matt
Well, the key to squeezing out all the performance you can is keeping the IO Subsystem busy. If you queued things to different threads using APC, that means you''d be waiting for the scheduler to make a context switch to the right thread to initiate the send. With IO Completion ports, generally I''d think that you would send from any thread you want, and just let the IO threads handle your IO Completion for sends.
Receiving would be slightly different, it has to be more planned, since when you call WSARecv, you''re really just supplying a buffer to be used when it''s needed most of the time, but there could also be waiting data so your WSARecv might complete right away. You''d probably want to figure out some ideal number of receive buffers you''d like posted when you create the socket, allocate them and call WSARecv on them, then just rotate those buffers through calls to WSARecv. That way you don''t have memory allocation / thrashing issues for your buffers.
Receiving would be slightly different, it has to be more planned, since when you call WSARecv, you''re really just supplying a buffer to be used when it''s needed most of the time, but there could also be waiting data so your WSARecv might complete right away. You''d probably want to figure out some ideal number of receive buffers you''d like posted when you create the socket, allocate them and call WSARecv on them, then just rotate those buffers through calls to WSARecv. That way you don''t have memory allocation / thrashing issues for your buffers.
What I''d like to know is whether it''s ok to post additional overlapped I/O requests from inside your callback. Jeffrey Richter wrote in an article on MSDN that you can''t, because the non-IO thread which calls your BindIoCompletionCallback routine could be terminated with your outstanding I/O still pending. The platform SDK contradicts this, stating that both I/O and non-I/O threads can initiate I/O requests, and that neither would be terminated with I/O still pending. Any clarification on the matter?
Matt
Matt
Read it more carefully:
This description really isn''t that clear but it essentially says, "If you post an overlapped I/O request to a non-I/O work thread, if the operation doesn''t complete in a timely manner, the thread might terminate and all pending overlapped operations will be cancelled." That is my interpretation of the information with a bit of help from Jeffrey Richter
The idea is that you receive I/O completion notifications on non-I/O threads but you queue I/O requests to I/O threads using QueueUserWorkItem(). When the work item is processed in the I/O thread, an overlapped I/O request is made against the socket/file handle. The call to the asynchronous function returns immediately and allows the I/O thread to process more work items. Since you bound the socket handle to the thread pool, when the overlapped I/O operation completes a callback is made on a non-I/O thread. Then the process repeats.
Hope this clears things up.
Regards,
Dire Wolf
www.digitalfiends.com
quote:
From Platform SDK, August 2001 Edition
A non-I/O worker thread waits on I/O completion ports. Using non-I/O worker threads is more efficient than using I/O worker threads. Therefore, you should use non-I/O worker threads whenever possible. Both I/O and non-I/O worker threads do not exit if there are pending asynchronous I/O requests. Both types of threads can be used by work items that initiate asynchronous I/O completion requests. However, avoid posting asynchronous I/O completion requests in non-I/O worker threads if they could take a long time to complete.
This description really isn''t that clear but it essentially says, "If you post an overlapped I/O request to a non-I/O work thread, if the operation doesn''t complete in a timely manner, the thread might terminate and all pending overlapped operations will be cancelled." That is my interpretation of the information with a bit of help from Jeffrey Richter
The idea is that you receive I/O completion notifications on non-I/O threads but you queue I/O requests to I/O threads using QueueUserWorkItem(). When the work item is processed in the I/O thread, an overlapped I/O request is made against the socket/file handle. The call to the asynchronous function returns immediately and allows the I/O thread to process more work items. Since you bound the socket handle to the thread pool, when the overlapped I/O operation completes a callback is made on a non-I/O thread. Then the process repeats.
Hope this clears things up.
Regards,
Dire Wolf
www.digitalfiends.com
3000-5000 connections per hour is a miniscule amount of connections. For this appliation it doesn''t matter how you handle connections...(assuming they won''t all be active at once). Even creating a new thread for each connection will work in this case. Just code it in perl for that matter. Bothering with I/O completion ports for this is a waste of time unless your traffic is going to grow a hundredfold.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
