I am in the middle of building up the basic structure of some network code I''m working on. It is based in C#, using UDP sockets for a reliable UDP layer. Right now I''m trying to figure out what the best way is for handling messages from the user to the network layer and from the network layer to the user. My thinking is something like this: +++++++SENDING+++++++ The user will initiate a Send() call. That Send() call''s data will be put into a UserSendBuffer. The idea behind this buffer is that I keep the Send() call from blocking. From this point, a continuously looping thread, will pull the Send() call''s data from the UserSendBuffer and will be processed by its appropriate connection state, housed in a hashtable. A datagram is created by assigning flags, a sequence number, acknowledgement number, etc. Once processed, the connection state will put the newly created datagram into a ProtocolSendBuffer. Another continuously looping thread will pull from the ProtocolSendBuffer where it will be sent out the socket using Socket.SendTo(). +++++++RECEIVING+++++++ The continuously looping socket thread will receive a datagram and insert it into the ProtocolReceiveBuffer. The continuously looping connection thread will pull from the ProtocolReceiveBuffer, process it for acknowldgements, sequence numbers, etc. Then a Receive() event is fired for the user to do what they wish with the data. ++++END+++ Am I getting too complicated with the buffers? Should I use a different architecture? maybe where I use a threadpool instead of the 2 continuously looping threads that handle the Connection States and the Sockets? How could i use a threadpool instead in this case? Any comments, suggestions are greatly appreciated Thanks, synth0id

Threads cause synchronization bugs, and synchronization cost. Avoid wherever possible.

Here''s an alternate design:

1) user can allocate buffers from the networking layer
2) user can write data into buffers
3) user can queue buffers (as messages) for sending
4) networking layer may optionally coalesce multiple buffers into a single packet -- that''s transparent to user
5) user is responsible for calling Network:oll() in the main loop

Inside Network:oll(), the network will:

1) send out any pending messages (or at most X bytes of pending messages, if you rate limit)
2) poll sockets for incoming messages
3) for any socket with incoming messages, read out the message from the socket
4) for any received message, dispatch to the user using a registered per-message-kind callback (delegate in C#; interface class in C++)

This way, the user knows that all actual networking will happen inside Network:oll(), which is very useful from a structure point of view.

An alternative to 4) would be to keep the messages around, and have the user ask for incoming messages in the main loop. Which you prefer is up to you. But if you don''t do callbacks, the user may forget to ask, and a large queue of undelivered messages may build up.

Ultimately though, won't the user end up implementing threads anyways?

If a user is filling the buffers, and at the same time has the Network.Poll() looping, won't they have to synchronize the buffers so that the user doesn't write the buffer while the Network.Poll() is reading the buffer and vice versa?

Or are you suggesting an entirely procedural approach, where a user writes to the sendBuffer, then reads from the recvBuffer, then calls poll() in that order, or similarly? Basically Poll() is never being looped by itself in its own thread? Is there any performance penalties to this approach once the number of virtual connections increases? It doesn't sound very scalable.

I'm liking the sound of the procedural approach, since I'm using the Poll() method to check the sockets, giving the user the same type of interface would be nice

[edited by - synth0id on April 11, 2004 3:35:53 PM]