11 replies to this topic

[xynapse]

this is something I've been through a few times and after a short break I am coming back with an application for a client, but just want to remember a few things before I actually implement anything.
I will be speaking about a multi-client server here, that deals with clients connecting through TCP onto a server bound port.
I remember coding this by using select() , FD_ISSET, FD_ZERO, FD_SET - and it's family methods to accept and manage a list of connected clients on my server - and I remember it was working great, but this time it is about a simple 32 player RPG game - so nothing fast-paced, thus the TCP here.
Simple GO_TO packets, WEATHER change, TIME updates, TEXT messages, ITEM STATE CHANGES and QUITS, LOGINS ( broadcasts ) , and something else I can't find out now.
What stops me from sleeping today is what should I really choose to have this server running smoothly ( when speaking of it's networking performance ).
People tend to speak much about 'non-blocking' approach to sockets either by using ioctlsocket(..) or some other 'threading' weirdos, which do the thing but you have to controll them in a different style - and yes I have tried this, and have seen many approaches - which deal with many threads around in the application - what is kind of overcomplicated really..
I have seen those MSDN "ASYNCHROUNOUS" sockets too, where you get the messages from the socket - but this is too much I think.
So instead of coding I am thinking and would like to ask,

• 
  if select() can do the trick when speaking of a game server, handling let's say those 32 players simultanously. What I mean here is how do you think from your experience, will that do the thing? Or I would have to rollback at the end and modify the whole server code while closing to the deadline..
  
• 
  what exactly is the difference between doing select / ioctlsocket ?
  
• 
  what are the pros and cons for both methods ?

Many many thanks for any reply to this, I hope I am clear in what I am asking about.

[Evil Steve]

1. For 32 players, I would use select() since you can easily get all 31 (worst case) sockets in the same fd_set.
2. Setting the socket to non-blocking and issuing a recv() on the socket will return 0 bytes if no data is waiting instead of blocking until data arrives, so you don't need to use select() to find out. However, select() is still useful for polling all the sockets at once.
3. First, stay away from async sockets - that turns into a bit of a mess . If you're already comfortable with threading, then you may want to have one thread that performs all socket operations (select(), send() and recv()). That way, any time that is spent making network calls can be absorbed. If you're not completely happy with threading, then I'd go with using select(). For a low number of sockets (<= FD_SETSIZE - which defaults to 64), it's nice and simple to use, and pretty quick to see if a socket can send or recv yet. To be honest, I'm not entirely sure where you'd want to use ioctlsocket to set a socket into non-blocking mode. Perhaps if you're only managing a single socket it may be quicker to try and recv() in non-blocking mode than to select() and find the socket doesn't need read. *shrug*.

[xynapse]

Evil, thanks for replying.

So far i've been doing with select and it really worked well - the reason i am asking is that people speak much about the non-blocking kinda stuff and i even tried that, but handling thread synchronisations is a bit mess for a game that deals with 32 players only. Sure it can be done, but it's like overcomplicating thing for me - or probably i haven't seen any good tutorial describing this.

The reason i am asking this is because my Client requested a project of a "small" 32 players RPG game - classic Click & Go, Click & Look, Click & Start fighting - thus it won't require a continous flow of packets flying around between them, and that's why i was thinking Select would do.

Now knowing select() would be choosen, let's make a small scenario here ( the worst one - patching ):

- Client connects to server sending authorization data and his data file version (this is a rar file, we are using vfs based on rar in our game) - this is around 1mb large
- Server checks for login/pass if ok, checks for latest data file version in MySQL DB
- If client's data file version is older, patching starts.

Now remembering we're select()ing here, how would that work:

- When patching is to start, Server sends back packet data to client saying: HERE IS YOUR NEW DATA FILE, here is the packet length and data file length - start sending
- Client recvs till the end.

Wouldn't that block server to send/recv to other clients already connected?

[Evil Steve]

For patching, that's slightly different. Yes, send() can block if you're sending a large amount of data (Which you probably don't need to worry about during the game in your cace), so for patching it might make sense to use ioctlsocket() to set the socket into nonblocking mode. Then, you can check the number of bytes written by send(), and try to send() the remaining bytes on your next update loop.

[xynapse]

Well , and this is the info i was looking for.

This means ioctlsocket can be called anytime to set non/blocking socket mode ? - in this case it would be called on client socket when patching starts, and when finished, comes back to blocking one - utilizing select?

[Evil Steve]

Yes. However, it might make your setup simpler if the patcher just sends the client the patched data and then closes the connection. The client can then open a new connection once it's patched, and the server can treat it as a whole new connection.

That also gives you the ability to patch the client's EXE file (With a bit of fudging), and to have the patch server a different executable on a potentially different machine to the game server.

[xynapse]

Yep, you're kinda right - but to clarify the approach, this is the actual protocol draft i'll be implementing.

CLIENT -> Connects to server, server accepts connection, sets timeout flag on this connection.
CLIENT -> Send auth data ( login/pass ) + RAR file version (VFS)
SERVER -> Check for authorization within SQL
SERVER -> If OK, check for version within SQL

SERVER -> if OK, send AUTH_OK, and all the other stuff, consider this client online and playing - this ends, and main work gets done on this client socket

SERVER -> if RAR version != version defined in database it is time to inform client about this
SERVER -> Send AUTH_OK - meaning authorization ok
SERVER -> Send UPDATE_START packet - meaning, you are now to be updated with new RAR file
-> UPDATE_START holds total file size of the RAR file, so client is aware how much to get

CLIENT -> Got AUTH_OK
CLIENT -> Got UPDATE_START - take the length of the file we are about to receive
CLIENT -> while(bytesreceived < UPDATE_RAR_FILE_LENGTH ) recv(...)
CLIENT -> GOT UPDATE_FINISHED, relogin with the same credentials

SERVER -> keep send()ing untill RAR size + packet headers are sent
SERVER -> UPDATE_FINISHED
SERVER -> close client connection


Now as long as client recv() is something i don't bother - cause it can block during recv,
i am a bit affraid of that send on the server side,
would that mean i have to call ioctl to unblock the socket right before sending UPDATE_START and block it again after file has been sent?


Sorry for bothering but just want to elaborate on this a bit as i am starting to code this today.

[Evil Steve]

xynapse, on 07 February 2012 - 06:45 AM, said:

would that mean i have to call ioctl to unblock the socket right before sending UPDATE_START and block it again after file has been sent?

You can call ioctlsocket() at any point, it makes most sense to do it just before doing the send()ing of the RAR file, and to re-enable blocking mode after.

[xynapse]

Seems clear,
what happens just right after i call ioctlsocket to unblock the socket?

How would i have to approach this knowing that let's say client socket = m_pClient->Socket;
Do i have to unblock server socket ( the one i bind to the port ) or client socket ( the one that i use to communicate with client )

Would i have to do it somehow like this ( pseudocode )

switch(pPacket->GetType())
{
  case ePackets::AUTH:
  {

      // Check if authorization is ok
      ....
     
     // Send first UPDATE packet consisting of the RAR file length
     server->send(UPDATE_START,m_pClient->Socket);

    // Enable non blocking mode on client socket ( or on server socket ? )
     ioctlsocket(m_pClient->Socket,nonblocking);

    // go into while
     while(bytessent<bytestosent)
     {
       server->send(UPDATE_DATA,m_pClient->Socket);
     }
     //unblock the socket
     ioctlsocket(m_pClient->Socket,block);

     //close client connection
     server->close(m_pClient->Socket);
  }
  break;
}

Or there is another better approach ?

[Evil Steve]

You call ioctlsocket on the socket you want to change - i.e. the client socket, since that's the one you're sending on. Also, the above code will need modified so you service other clients in the while() loop, you don't want to be doing an internal loop like that, you probably want a state machine to handle the client state.

[xynapse]

Ok i think i've got it, will post here some results.

Thanks Evil Steve.

[hplus0603]

I would not depend on ioctlsocket() at all. I suggest building a "reactor" for your networking, where you call select() on all the sockets, and then deal with each of the sockets by calling recv() and send() as appropriate. This is guaranteed to be non-blocking, because the first call to recv() is guaranteed to not block when select() says there is data.
The data would only be read into (or written from) some per-socket buffer. Then, as signal/event/message is sent to the rest of the application to tell it that there's more data in the buffer. The next layer would be the packet framing layer, which would figure out whether there's enough data to constitute a whole packet, and then decode/remove the packet and dispatch to a packet dispatch layer.
The packet dispatch layer is the thing that switches on packet type, and perhaps packet address, and dispatches to application logic.

Same thing for writing/sending -- data goes into an output buffer, and send() sends out of this buffer when available. If the output buffer backs up more than you are prepared to accept, then the client is hopelessly behind and should be dropped.

If you want to support large file transfers over the same system, it's pretty easy -- you just make the "there is space available" callback/signal/message fill the output buffer with more data/packets until you're done.

Now, the nice thing with this architecture is that if you ever want to scale up to more players, you already have the reactor/callback/dispatch infrastructure in place, so swapping to something like boost::asio is mechanically easy. However, your world update loop will probably be single-threaded in the simple case, and for thousands of players, at some point, you need to support sharing across multiple threads/cores -- but that's a different problem :-)