I have just never done networking coding before and want to understand the API and all the little details that go into the big implementation first. Very important for a solid foundation, and a solid foundation is very important for my task (a next-gen commercial game engine).

If this is the case, you will definitely want to look at existing UDP libraries used by games to get an idea of what they do and what features they provide. Namely, RakNet (complete game library), enet (low level wrapper), lidgren, NetDog (caters to MMOs), SmartFoxServer (a whole platform), Photon (another platform), boost::asio (cross platform wrapper library), POCO (similar to boost libraries), and finally ACE (toolkit).

Focus on the big picture first. The "little details" come about from the specific network implementation you use. I.e., using regular Winsock, POSIX, BSD sockets come with their own little quirps and platform specific limitations, so if your overall goal is a next-gen commercial game engine, then the platforms you will support will play a large part into what you might want to build upon.

For example, if you have no plans for console support, then there is no reason not to start looking over boost or poco which contain a lot of code dealing with the little things already. The research and development has been done, so you don't need to discover it all yourself as much as just look at existing solutions to see what is done. That is not to say you can't code your own implementation, you can if you so choose, but trying to relearn decades of R&D from scratch is not worth it.

If you do plan on supporting consoles, then things like boost or poco are no good to you, so you already know there's no real need for them because you don't want different networking cores for different platforms (imo). You can still consult them for PC specific stuff, but you'd probably want to go for something a lot more lightweight depending on all the languages you will have to support for your engine.

This article suggests not using a second thread to handle networking.
http://trac.bookofho...GameProgramming
He says it is tricky to get working while avoiding race conditions, deadlocks, etc.
These things are non-issues at my level. Should I still avoid threading? To me I think it would simplify a few things and make some things smoother, but again my knowledge of networking is naive, and I worry about getting bitten in the ass by some unforeseen hiccup.[/quote]

That unforeseen hiccup is generally " race conditions, deadlocks, etc.". "Stuff" happens, even when you are familiar with the concepts.

All it comes down to is "shared state" really.

This is more of a bigger issue with TCP than it is with UDP. Since TCP is a stream protocol, if you have multiple recvs posted on a socket, you must maintain the order that the data is returned before processing it. So if thread 1 recvs data into a buffer object with a sequence id of N and thread 2 recvs data into a buffer object with sequence id N + 1, then you must be sure to process the object with id N before N + 1. This means you have to code a custom system to handle this if you need that type of functionality. Unless you are working with massive data transfers using hardware that is meant for it, then usually it's just easier to keep one recv posted on a socket at a time and only process data from one thread. That one thread might be a worker thread in a pool of many, so it may not be the same each time, but only one concurrent recv is posted at a time.

Let's say you are using TCP and doing it the traditional way. Each connection has a context that has a receive buffer. You only have one concurrent recv operation in progress at once, so no other thread will access the buffer. Once you receive data into the buffer, you parse out your packets.The trouble now is what do you do with the packets? Typically, you lock a context specific queue and add them so they can be processed elsewhere. In the thread that will process them, the queue is locked, copied into a new queue, cleared, then unlocked. This is to help reduce lock contention between the network thread and the thread processing packets. The thread that is processing the packets does not run as fast as the network thread, so "bursts" of packets do not require a lot of lock contention. I.e., checking for packets hundreds-thousands of time per millisecond does not make much sense (in the context that we are talking about, there are other systems where it might).

Other designs dispatch the packet processing logic from the network thread. This design only works for systems with very little shared state (as to not tie up the network thread) and very lightweight logic processing. For example, a simple web server with no shared sate between the connections might take this approach. Since the context object will not be accessed from any other part of the system, it works out 'ok'. If you were trying to write a MMO, then this design is not practical because you tie up a network worker thread (which if your thread pool does not grow, can be a real problem). It's far easier to deadlock the threads with a lot of shared state locks so by keeping them to a minimal (i.e. packet queue lock then dispatch) you reduce the problems you might have.

With UDP, you generally only need one main thread to handle the server socket. This is because you are not working with "connections" like you are in TCP. As a result, when you get a packet in the network thread, you can simply pass it to another thread to take care of processing it. In other words, the network thread is only responsible for pulling packets from the wire and passing them along. For this to work out, you have to make sure the method you are using to pass the packets along does not involve a lot of overhead.

For example, let's say you recvfrom one packet. You lock the queue, add the packet, and continue on. If the thread that is processing the packets acquires the lock but does not release it fast enough by the time the next packet is processed, then you stall the network thread some each time and really don't gain anything from this setup. That is why, going back to the TCP example, when you lock the queue, copy it, and release, you minimize the contention and can maintain a well functioning system in that regards.

Depending on the platform and language, you might have to optimize the "copy" step to make sure you don't get hit with unneeded global locks for allocation or deep copying. Even so, that's an optimization that only should be pursued after profiling and having solid evidence it's causing issues! Likewise, the lock mechanisms you use do matter. If you are on a platform where the most viable locks incur significant overhead, then the benefits of using multiple threads is greatly reduced.

Once you managed to get your packets out of the network layer, then you are back to dealing with your typical issues of message validation, multi-threaded programming, and so on.

Working out how you "send" data through the network might be more troublesome with some designs then others. For example, you never want to allow threads to arbitrarily send data instantly. If you do, you just make life significantly harder than you have to when things go wrong. Instead, you want to save outgoing messages to a queue and process them from a specific context. The implications of doing this though can be quite complex.

For example, with TCP, you have to wait until all the previous buffered data is passed through send before starting the next send. As a result, you must make sure you chain your send triggering logic so you don't end up with data still in a send buffer waiting to be passed through send, but no event is pending to trigger it. This is more related to the multi-threaded programming issues than just networking. With UDP, you just have to make sure that up until the packet data is dispatched with sendto,you maintain protocol synchronization so you don't have a case where you send a packet with sequence number N, but a security flag based on sequence number N + 1 as a result of not handling your locks the right way (i.e., locking individual pieces of logic instead of the entire unit). If you don't have any extra protocol specific security stuff, then it's not as big of an issue, but any decent system should.

Of course, if you are only using one thread and very simple networking APIs, like select, you don't have all the headaches to deal with when it comes to multi-threaded programming, so keep that in mind. I'd say start simple and small and see how it carries you. I've found it's nice to have solutions for both single and multi-threaded setups, so don't feel like you have to just dive into making an uber multi-threaded solution whose power and functionality might not ever be utilized due to other constraints. In the end, a lot of what you do is going to depend on your target platforms and the language you use.

Lastly, since I couldn't fit it in anywhere else, you have to keep in mind the data type differences and byte order of different platforms and architectures. You must code your higher level network logic to take this into account. Otherwise, you will run into tons of (not so) fun issues that are really hard to figure out after the fact. So keep in mind: size_t size differences, wchar_t size differences, float/double precision differences, code page string processing differences, and many more. A lot of the things you have to be worried about are not directly related to network API issues in network programming!