• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
Sign in to follow this  
Followers 0
Figgles

Linux epoll vs non-blocking sockets

13 posts in this topic

Has anyone tested the scalability of non-blocking sockets each being either read(2)'d or recv(2)'d versus Linux 2.6's epoll(7) interface? Is there a similar interface for Solaris 10/SPARC (6/06)? I am looking for the CPU usage at 100, 1000, and 5000 file descriptors. Code: I current have a listening thread and a number of "worker" threads. The listener thread finds new connections, marks them as non-blocking, and handles off the file descriptor the worker thread with the fewest current connections. The worker thread's loop is essentially: for( number of file descriptors 'i' ) { int s = filedescriptor[i] if(read(s, &buffer, size) > 0)) { /* Verify packet, etc. */ } } Hardware: I am running this on a quad processor UltraSPARC server with 4GB of RAM. The server does fairly computationally expensive work on each thread, save the listen thread, so single threading is a non-option -- why have four processors if you aren't going to use them? I think that an interface such as epoll(7) would reduce the total number of userspace->kernelspace transitions as it could test the entire array of file descriptors as the same time, but I have heard on and on that non-blocking I/O + read(2) or recv(2) is the way to go. Considering that I am trying to make a real-time responsive server, and preserve as much CPU time as possible for the requests, is epoll(7) a good idea? Is there a better mechanism under Solaris 10/SPARC (6/06)? Does anyone have some personal experience with the scalability of such things?
0

Share this post


Link to post
Share on other sites
Calling recv() on 5,000 sockets sounds like 5,000 trips into the kernel, which is probably not ideal, unless you know that almost all sockets will have new data almost all of the time.

Just because you have computation in threads, doesn't mean that the receiving needs to be multi-threaded. You can, for example, receive in a single thread, using epoll or similar, and farm out work to the worker threads. I don't know which method would be better on Solaris -- why don't you implement both and measure?

Also, you're not guaranteed to get a whole packet (or only one packet) in the call to read(). Hopefully, you're dealing with partial packets in the "etc" branch.
0

Share this post


Link to post
Share on other sites
Farming out the work to other threads seems like a headache from a synchronization standpoint, requiring large amounts of memory being marked as shared, either by setting up a work queue or actually inserting the information to be computed. And it would at best give the same performance that I am getting now, which requires no synchronization in the best case. This of course assumes that the cost of synchronization is greater than the cost of recv(2). While I am not for sure this is true, it seems highly likely, due to the memory traffic involved in copying data around and threads awaiting work and sitting still in between. I was asking more about the scalability of epoll(7) on Linux, and as I found out later, /dev/poll on Solaris.

Yes, I know packets can become fragmented, or rather, not fully recv(2)'d, that was just an overly simplified piece of code.

Please stay on topic: Does epoll(7) under Linux scale better than recv(2) on non-blocking sockets? Any information on Solaris 10/SPARC (6/06)?
0

Share this post


Link to post
Share on other sites
I don't know the answer to your specific question: I suggest you benchmark it yourself.

However, I would like to submit that I disagree with your assessment of what threading means for performance in this case. So that future readers do not get mis-led, my belief is that there is no more copying necessary of data in the threaded case, and synchronization for a thread-specific work queue is typically significantly cheaper than entering the kernel for an I/O primitive like recv().
0

Share this post


Link to post
Share on other sites
Quote:
Does epoll(7) under Linux scale better than recv(2) on non-blocking sockets?


lol you can't compare epoll with recv on non-blocking sockets. You need all 3 things together.
0

Share this post


Link to post
Share on other sites
True, one still does need to use recv(2) or read(2) to actually read the data from the socket. However, what I am refering to is using epoll(7) as a method of determining whether data is present, rather than polling the sockets with a call to recv(2) rapidly and getting an error that data is *not* present.

I think hplus0603 is correct about the userspace->kernelspace transition from a large number of I/O operations probably will use more time merely context switching than getting work done. I am going to benchmark this soon, if anyone would like the results, I can post them here when I am finished.
0

Share this post


Link to post
Share on other sites
You typically use EITHER non-blocking sockets OR epoll.

With epoll (or select), you get told about sockets that you can safely recv on without blocking. Thus, you can call recv once, even if the socket is blocking.

With a non-blocking socket, you typically call recv on the socket once through each loop in your game, which is OK for small amounts of sockets (and also for UDP-based servers, that typically only have a single socket), but not for TCP servers with lots of clients.
0

Share this post


Link to post
Share on other sites
Quote:

With epoll (or select), you get told about sockets that you can safely recv on without blocking.


On Winsock newsgroups experts say that this is not true. It can still block.
0

Share this post


Link to post
Share on other sites
winsocks is diffrent from sockets winsocks was written as a compatibality layer between windows networking and porting *nix code over to windows
0

Share this post


Link to post
Share on other sites
Quote:
On Winsock newsgroups experts say that this is not true. It can still block.


If select() returns a socket as readable, then recv() will not block on the next read, guaranteed.

This is not true for certain Windows-specific versions, like the infamous WSAAsyncSelect(). Stay away from AsyncSelect().

0

Share this post


Link to post
Share on other sites
When 'select' says readable, that means it *was* readable at the time 'select' said it was. It has *never* provided a future guarantee of readability. *No* status reporting system call provides future guarantees.
0

Share this post


Link to post
Share on other sites
You can't compare recv() and epoll they exist for different purposes. epoll is an efficient facility for determining which of a large number of file descriptors has stuff waiting to be read/ written / errors etc.

Of course epoll is vastly preferable to 5000 recv()s if almost all of them return EWOULDBLOCK (e.g. there is no data waiting and they're non-blocking).

But if you know that there are data to read, you may as well read them.

epoll is a Linux-specific facility (although some other Unixes have similar ones) which enables you to add a FD to watch, just once, and continue to watch it until you tell it to stop (or you close it).

Then each call to epoll_wait will wait for any of the things the epoll FD is currently configured to wait for, which could be a great many. This saves the setup work required for select().

Moreover, you can get an array back telling you exactly what has happened on which watched FDs.

Mark
0

Share this post


Link to post
Share on other sites
Quote:
When 'select' says readable, that means it *was* readable at the time 'select' said it was. It has *never* provided a future guarantee of readability. *No* status reporting system call provides future guarantees.


Not true. Once data is queued for the incoming socket, then that file descriptor WILL be readable, until you close that socket. If select() returns a given socket as readable, it WILL NOT BLOCK the next time you call recv(), unless you take explicit action on that socket in the meanwhile.

There may be other kinds of file descriptors (not sockets) where this is not true. However, the interaction between select/recv() is defined to be exactly this, for good reason. Also note that I'm talking about recv(), not read().
0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0