Linux epoll vs non-blocking sockets

Figgles · 2007-01-23T23:38:33

Has anyone tested the scalability of non-blocking sockets each being either read(2)'d or recv(2)'d versus Linux 2.6's epoll(7) interface? Is there a similar interface for Solaris 10/SPARC (6/06)? I am looking for the CPU usage at 100, 1000, and 5000 file descriptors. Code: I current have a listening thread and a number of "worker" threads. The listener thread finds new connections, marks them as non-blocking, and handles off the file descriptor the worker thread with the fewest current connections. The worker thread's loop is essentially: for( number of file descriptors 'i' ) { int s = filedescriptor if(read(s, &buffer, size) > 0)) { /* Verify packet, etc. */ } } Hardware: I am running this on a quad processor UltraSPARC server with 4GB of RAM. The server does fairly computationally expensive work on each thread, save the listen thread, so single threading is a non-option -- why have four processors if you aren't going to use them? I think that an interface such as epoll(7) would reduce the total number of userspace->kernelspace transitions as it could test the entire array of file descriptors as the same time, but I have heard on and on that non-blocking I/O + read(2) or recv(2) is the way to go. Considering that I am trying to make a real-time responsive server, and preserve as much CPU time as possible for the requests, is epoll(7) a good idea? Is there a better mechanism under Solaris 10/SPARC (6/06)? Does anyone have some personal experience with the scalability of such things?

Networking and Multiplayer Programming

Started by Figgles January 10, 2007 08:19 PM

12 comments, last by hplus0603 17 years, 2 months ago

hplus0603

11,916

January 22, 2007 08:32 PM

Quote:On Winsock newsgroups experts say that this is not true. It can still block.

If select() returns a socket as readable, then recv() will not block on the next read, guaranteed.

This is not true for certain Windows-specific versions, like the infamous WSAAsyncSelect(). Stay away from AsyncSelect().

enum Bool { True, False, FileNotFound };

Anonymous

January 23, 2007 02:21 AM

When 'select' says readable, that means it *was* readable at the time 'select' said it was. It has *never* provided a future guarantee of readability. *No* status reporting system call provides future guarantees.

markr

1,692

January 23, 2007 03:17 AM

You can't compare recv() and epoll they exist for different purposes. epoll is an efficient facility for determining which of a large number of file descriptors has stuff waiting to be read/ written / errors etc.

Of course epoll is vastly preferable to 5000 recv()s if almost all of them return EWOULDBLOCK (e.g. there is no data waiting and they're non-blocking).

But if you know that there are data to read, you may as well read them.

epoll is a Linux-specific facility (although some other Unixes have similar ones) which enables you to add a FD to watch, just once, and continue to watch it until you tell it to stop (or you close it).

Then each call to epoll_wait will wait for any of the things the epoll FD is currently configured to wait for, which could be a great many. This saves the setup work required for select().

Moreover, you can get an array back telling you exactly what has happened on which watched FDs.

Mark

hplus0603

11,916

January 23, 2007 11:38 PM

Quote:When 'select' says readable, that means it *was* readable at the time 'select' said it was. It has *never* provided a future guarantee of readability. *No* status reporting system call provides future guarantees.

Not true. Once data is queued for the incoming socket, then that file descriptor WILL be readable, until you close that socket. If select() returns a given socket as readable, it WILL NOT BLOCK the next time you call recv(), unless you take explicit action on that socket in the meanwhile.

There may be other kinds of file descriptors (not sockets) where this is not true. However, the interaction between select/recv() is defined to be exactly this, for good reason. Also note that I'm talking about recv(), not read().

enum Bool { True, False, FileNotFound };

Linux epoll vs non-blocking sockets

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Linux epoll vs non-blocking sockets

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines