• Create Account

## IO completion port

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

9 replies to this topic

### #1Laval B  Members

Posted 18 May 2012 - 09:22 AM

Hello everyone

I have started working on a server project. The server is being developped in C++ and will have to work on Windows and eventually on Solaris but we're focusing on Windows for now.

The server is basically some sort of proxy server that will use a home protocol to provide access to the data (database and file transfer) for applications. It will have to service a few hundreds clients.

I have read that, in order to scale, we should use asynchronous I/O. On windows platform, it means using I/O completion. For what i understant about it, evey time the server receives a connection it registers it to an I/O completion port "object". Then you can have a certain number of threads sitting on calls to

GetQueuedCompletionStatus which will wake the threads when an I/O operation completes. I am wrong ?

So, i will need a pool of threads to manage the I/O completion operations. I will likely need another pool to process the queries in order to keep the I/O from blocking for too long,

I was looking at the Win32 threadpool api. At a first glance, it doesn't seem too bad but is it really something i should use ? It seems like the threadpool api is also capable of handling I/O completion.

Also, does anyone have a good reference on I/O completion port ?

We think in generalities, but we live in details.

### #2Len Holgate  Members

Posted 18 May 2012 - 09:30 AM

I have some example IOCP code that you could download to play with. It provides some classes which wrap up the complexity somewhat. You can get the code from here: http://www.serverframework.com/products---the-free-framework.html There are also links to the code project articles that I wrote about this code back in 2002.

### #3Laval B  Members

Posted 18 May 2012 - 09:35 AM

I have some example IOCP code that you could download to play with. It provides some classes which wrap up the complexity somewhat. You can get the code from here: http://www.serverfra...-framework.html There are also links to the code project articles that I wrote about this code back in 2002.

Thank you very much. I'll take a look at it. Iocp can be a bit confusing when you look at it for the fist time.
We think in generalities, but we live in details.

### #4sjhalayka  Members

Posted 18 May 2012 - 10:05 AM

Perhaps I'm just lazy. It sounds like you're building something that produces results that are awfully close to what you'd get from a web service like SOAP, so why not just throw a copy of Apache and gSOAP or Axis2 onto your server and be done with it? Most decent RDBMSs include a C/C++ API, so there you go... a fairly secure middle tier up and running in less than a day, and you still get to maintain control at the source code level.

Yes, I know that socket programming can be extremely fun (challenging, rewarding), but I just thought I'd throw the web services route out there. I've done this before with Apache/gSOAP/MySQL Community Edition, and it all worked out extremely well.

P.S. Regarding binary data transfer: "Apache Axis2 supports Base64 encoding, SOAP with Attachments and MTOM (SOAP Message Transmission Optimization Mechanism)."

Edited by taby, 18 May 2012 - 10:25 AM.

### #5Len Holgate  Members

Posted 18 May 2012 - 10:29 AM

Thank you very much. I'll take a look at it. Iocp can be a bit confusing when you look at it for the fist time.

Yes, it can, but it's actually fairly straight forward once you get your head around the concept. Remaining async throughout your data processing is the more complex part but that's not too bad if you just build state machines.

The code that I linked to is stable and reliable but it is missing quite a few features that you probably should think about, especially if your server will be sending 'push' notifications to clients that may be on slow network links. I talk about the issues that you can come up against if you ignore TCP flow control with IOCP style async networking APIs here on my blog: http://www.serverframework.com/asynchronousevents/2011/06/tcp-flow-control-and-asynchronous-writes.html

### #6Laval B  Members

Posted 18 May 2012 - 11:55 AM

I talk about the issues that you can come up against if you ignore TCP flow control with IOCP style async networking APIs here on my blog: http://www.serverfra...ous-writes.html

Thanks again for all the useful info, it's appreciated.
We think in generalities, but we live in details.

### #7Kaelb  Members

Posted 21 May 2012 - 06:37 PM

I made a work for my Operative System class using IOCP for a server, at first it was pretty hard to understand, but when you get the idea of how it works you love it.

To learn a bit more about IOCP I used the book "Windows via C/C++", it has some good pages about it and why it is so good, and of course, an example with comments.

What you said about IOCP is correct, when I did my workd there was no need for a threadpool api, but I'm not really an expert in the windows API, so I can't really help you on that one.

One more thing that's very good about IOCP is that it can manage the number of threads running so you don't have to waste context switches and it tries to maximize the the workd made by the threads, instead of wasting processor time context switching between them.

### #8krippy2k8  Members

Posted 22 May 2012 - 06:29 AM

This is not a direct answer to your question, but as a suggestion you might want to consider looking at Boost.Asio (or non-Boost Asio). It's an asynchronous IO library that basically uses the best available methods on all supported platforms (which includes Solaris). On Windows, it uses IOCP. It includes a high-performance implementation of the highly scalable Proactor framework (and on systems without native asynchronous IO it will emulate it via a Reactor on the backend), and will free you from having to worry about most of the details and gotchas when working with IOCP at a low level.

### #9Laval B  Members

Posted 22 May 2012 - 09:09 AM

One more thing that's very good about IOCP is that it can manage the number of threads running so you don't have to waste context switches and it tries to maximize the the workd made by the threads, instead of wasting processor time context switching between them.

Yes, that's one thing i find interesting about it and i actually like the way it works. It should scale very nicely with the number of CPUs (cores) on machines.

we did take a look at SOAP like Taby said but considering the nature of what we're doing, it is not really appropriate. Don't get me wrong, those tools are really neat. I'll take a look at non-boost asio.

I have work on a small prototype and encontered a weird problem with my I/O worker threads. The server thread basically waits on an event that is triggered when a connection is received, her's the code for the thread routine :

int AsyncServer::Run()
{
bool bRunning		   = true;
DWORD iRet			  = 0;
HANDLE Events[SE_COUNT] = {0};
if(mIocp.Create(miConcurency))
{
if(mIoWorker.Initialize(&mIocp))
{
Events[SE_ACCEPT_EVENT]   = mhAcceptEvent;
Events[SE_SHUTDOWN_EVENT] = mhShutdownEvent;
// Reset the shutdown event.
ResetEvent(mhShutdownEvent);
if(CreateListeningSocket(BACKLOG_COUNT))
{
if(WSAEventSelect(mListener, mhAcceptEvent, FD_ACCEPT) != SOCKET_ERROR)
{
while(bRunning)
{
DWORD iWaitResult = WaitForMultipleObjects(SE_COUNT, Events, FALSE, INFINITE);
switch(iWaitResult)
{
case WAIT_OBJECT_0:
{
ResetEvent(mhAcceptEvent);
//TEST
DESCRIPTOR Socket = WSAAccept(mListener,
NULL,
NULL);
closesocket(Socket);
TRACE(L"\n--- Accepted connection ---\n");
//END TEST
}
break;
case (WAIT_OBJECT_0 + 1):
{
// Shutdown the server.
bRunning = false;
//TEST
TRACE(L"\n--- Server shutting down ---\n\n");
//END TEST
}
break;
}
}
}
}
// Server shutdown.
ShotdownSequence();
}
}
return 0;
}


The mIoWorker.Initialize does this :

bool bRetVal			  = true;
IOWorker *pWorker		 = NULL;

mpIocp					= pIocp;

for(unsigned int i = 0; i < iThreadCount; ++i)
{
pWorker = new(std::nothrow) IOWorker(pIocp);
if(pWorker)
{
mWorkers.push_back(pWorker);
bRetVal = bRetVal && pWorker->Start();
}
}
return bRetVal;


So, in order to stop the thread, the mhShutdownEvent event is signaled which breaks the loop and gets to the ShotdownSequence method. The method called to stop the server basically only signal the event then calls WaitForSingleObject on the thread handle.

The ShotdownSequence() method does the following :

void AsyncServer::ShotdownSequence()
{
// Shutdown the I/O thread pool.
mIoWorker.Shutdown();
// Close the iocp.
mIocp.Close();
mbIsRunning = false;
}


and mIoWorker.Shutdown() post a completion packet for each thread and waits for each thread to terminate :

void IOWorkerPool::Shutdown()
{
IOWorker *pWorker = NULL;
std::vector<IOWorker *>::iterator it;

for(unsigned int i = 0; i < mWorkers.size(); ++i)
{
mpIocp->PostIoCompletionOperation(NULL);
}
for(unsigned int i = 0; i < mWorkers.size(); ++i)
{
mWorkers[i]->Wait();
}
while(mWorkers.size())
{
pWorker = (*mWorkers.begin());
delete pWorker;
mWorkers.erase(mWorkers.begin());
}
pWorker = NULL;
}


The problem i have with this is that it freezes once in a while waiting on a thread (ramdom one) when it shutsdown. However, the threads all terminate normaly. If i put a Sleep(2000) as a test instead of the loop that calls mWorkers[i]->Wait(), everything terminate just fine (Sleep is just for a test). The wait method simply calls WaitForSingleObject on the thread handle. I really don't see what happens ....

The I/O workers only do the following so far :

int IOWorker::Run()
{
DWORD iBytes			 = 0;
LPOVERLAPPED pOverlapped = NULL;

while(mbIsRunning)
{
{
{
}
else
{
mbIsRunning = false;
}
}
}
return 0;
}


Edit :

Is it wrong to use WaitForSingleObject to wait for a thread to terminate (or at least to call it many times in row) ?

Edited by Laval B, 22 May 2012 - 10:43 AM.

We think in generalities, but we live in details.

### #10Laval B  Members

Posted 22 May 2012 - 03:45 PM

Ok, i found what the problem is and i thought i might as well mention it since it is related to the Win32 thread api. Alot of you probably know that already.

The problem was hiding in my thread class. I was using _beginthread to create the thread. Reading the Microsoft documentation i learnt that this function may not always return a valid handle (especially if the thread terminates too quickly) which can cause WaitForSingleObject to hang. One solution is to call endthread/endthreadex explicitly at the end of the thread function. A better solution is simply to use _beginthreadex which solved the problem. It also uses a thread function that returns a value which will be the return code of the thread (that can be forced with endthreadex).

Edit :

I would also like to add, @krippy2k8, the non-boost asio looks like a very nice library.

Edited by Laval B, 22 May 2012 - 04:20 PM.

We think in generalities, but we live in details.