Jump to content

  • Log In with Google      Sign In   
  • Create Account

Interested in a FREE copy of HTML5 game maker Construct 2?

We'll be giving away three Personal Edition licences in next Tuesday's GDNet Direct email newsletter!

Sign up from the right-hand sidebar on our homepage and read Tuesday's newsletter for details!


Threading and networking


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
11 replies to this topic

#1 phayer   Members   -  Reputation: 150

Like
1Likes
Like

Posted 17 March 2014 - 08:36 AM

So I'm stuck, and can't seem to figure out what the "right way" to solve this is.

 

I'm using enet - and the "enet_host_service" which takes care of sending and receiving data takes a timeout, like 10ms or 5000ms - whatever. The problem is; how large should this be? 1ms?

 

Cause the wait freezes the rest of the thread, of course. But if I have two threads, one that reads from the top of a list(simulation) and another one that writes to the bottom of the list(network) - wouldn't that impact performance hard and in some cases cause problems with locking?

 

Another thing I thought about was using zeromq(zmq) internally as I then don't have to think about locking, but I'm very unsure about zeromq when it comes to latency.

 

Any suggestions?



Sponsor:

#2 spinningcube   Members   -  Reputation: 90

Like
0Likes
Like

Posted 17 March 2014 - 08:51 AM

So I'm stuck, and can't seem to figure out what the "right way" to solve this is.

 

I'm using enet - and the "enet_host_service" which takes care of sending and receiving data takes a timeout, like 10ms or 5000ms - whatever. The problem is; how large should this be? 1ms?

 

Cause the wait freezes the rest of the thread, of course. But if I have two threads, one that reads from the top of a list(simulation) and another one that writes to the bottom of the list(network) - wouldn't that impact performance hard and in some cases cause problems with locking?

 

Another thing I thought about was using zeromq(zmq) internally as I then don't have to think about locking, but I'm very unsure about zeromq when it comes to latency.

 

Any suggestions?

 

I think what you suggested is the best. Have one thread taking care of network reads and writes (handling the Enet context) and one or however many you need for what you would like to do in terms of other processing. If you are handling messages with one thread then even if there is interlocking due to some queue issues, that should not be a big problem as it's not that heavy operation (locking the mutex) and you don't actually wait all that long as you have 1kHz at most reads from the list, and if you construct the read/write to be "instant" like I suggest below then there should be no issue of waiting.

 

I solved this by having an in process list (with messages to be sent) and one in queue list (with messages to be soon added to be sent). Like this you off-load the interlocking by only making the lock-unlock depend on a super fast pointer setting operation between two lists.

 

1 ms is good I think, depends on your latency issues.

 

PS - Also you might be too smart for your own good. You seem to be doing future-proof-development. Just wing it, implement it and see ;-)


Edited by jbadams, 15 April 2014 - 02:16 AM.
Restored post contents from history.


#3 hplus0603   Moderators   -  Reputation: 5508

Like
1Likes
Like

Posted 17 March 2014 - 09:47 AM

There are two main options:

1) Use a thread for networking. Set the timeout to infinity. Use some kind of queue between network thread and main (or other processing) threads. That queue could be non-blocking if you want, if you know how to deal with back-ups/overflows.

2) Poll networking in your main thread. Unless your main thread funnels all different timing events into a single dispatcher (clock, I/O, etc) then it's likely best to pass a zero timeout, to make it truly a "poll" rather than a "wait."
enum Bool { True, False, FileNotFound };

#4 fholm   Members   -  Reputation: 262

Like
0Likes
Like

Posted 17 March 2014 - 10:00 AM

If you want to see a source-code of an implementation of the 2nd solution that hplus described, you can check out my UdpKit networking library: https://github.com/fholm/udpkit

More specifically you can check out the chat example: https://github.com/fholm/udpkit/blob/master/src/managed/udpkit.example.chat/Program.cs as it implements this type of polling that hplus describes. All of this is in C# so maybe it doesnt apply for you.

#5 Tispe   Members   -  Reputation: 1038

Like
0Likes
Like

Posted 18 March 2014 - 07:32 AM

I am wondering about this myself. I use a thread which Enet is created and managed on. The whole thing boils down to 1ms latency vs 100% CPU usage.

 

In the networking thread I can set the timeout to 1ms, such that it will use less then 1% of the CPU. But this may cause 1ms latency, or if Sleep() is used inside the Enet library upto 10ms iirc. I have only found some while loops in the Enet code, I can't figure out how it waits for packets, maybe it has driver interupts waking it up. Which eliminates the latency, but idk.

 

Or have a 0ms timeout and the thread will consume 100% of the core it is running on. So I don't know which to use to be honest.

 

But anyway, I don't think one should have more then 1ms timeout. Since your main thread needs to send packets too. If you have a 5000ms timeout then you only send out packets every 5seconds...

 

I don't think Enet has internal threads for recieving/sending packets. Because if you don't service the host within 15-20seconds it will disconnect from the server. So if you have it in your main thread and the game freeze for 20sec then you disconnect.


Edited by Tispe, 18 March 2014 - 07:36 AM.


#6 hplus0603   Moderators   -  Reputation: 5508

Like
0Likes
Like

Posted 18 March 2014 - 11:40 AM

this may cause 1ms latency


If you have a 5000ms timeout then you only send out packets every 5seconds


I would expect Enet to return immediately if there is any data received, not wait the full time-out.
Thus, as long as data is coming in, there is zero latency between received data and getting the data in the program (for the first piece -- the next piece gets whatever latency your processing imposes.)

Now, for the outgoing data, you will need some kind of maximum send rate. One packet 20 times a second? That would mean you should set the timeout to no more than 50 milliseconds.
enum Bool { True, False, FileNotFound };

#7 Tispe   Members   -  Reputation: 1038

Like
0Likes
Like

Posted 19 March 2014 - 02:00 AM


Now, for the outgoing data, you will need some kind of maximum send rate. One packet 20 times a second? That would mean you should set the timeout to no more than 50 milliseconds.

 

I disagree. Suppose your enemy is using a spell on you or some other action. You want to get that packet ASAP. For position updates 50ms between each packet is fine.
 
You don't want Enet to manage how often you send packets. You want Enet to send a packet as soon as you tell it to. You can't have Enet waiting 50ms on another thread while an outgoing packets is waiting to be sent. If you are "lucky" and get an incoming message in those 50ms then your waiting is interupted and you can send that outgoing packet, but be unlucky and that outgoing packets sits in a queue for 50ms.
 
You want another layer ontop of Enet to decide which packets are sent and at what frequency. And thus Enet must be very responsive for any requests, be it a position update or an action.
 
So, 1ms (which can mean upto 10ms if Sleep() is used inside Enet) or 100% CPU usage....

Edited by Tispe, 19 March 2014 - 02:00 AM.


#8 phayer   Members   -  Reputation: 150

Like
0Likes
Like

Posted 19 March 2014 - 01:57 PM

Okey, so I implemented something that seems to work well;

 

2 threads and 2 queues. 

1 thread for handling networking, 1 for the game.

1 queue for incoming data, 1 for outgoing data.

 

Does this seem like a reasonable design? I don't want to get hung up on this, but on the other hand I don't want to make a lot code smell -- Like, I don't want to spend a month later rewriting the entire network only because my prototype implementation was bad.



#9 frob   Moderators   -  Reputation: 22205

Like
0Likes
Like

Posted 19 March 2014 - 02:20 PM

Other than to beware of concurrent edits (e.g. network thread adding to the queue while the processing thread is removing from the queue) yes, that can work quite well. The network is just a data transfer medium. You are right to want to keep it simple. Write data, read data, that's it.

 

The interesting part is what you do with that data inside the game.


Edited by frob, 19 March 2014 - 02:25 PM.

Check out my book, Game Development with Unity, aimed at beginners who want to build fun games fast.

Also check out my personal website at bryanwagstaff.com, where I write about assorted stuff.


#10 phayer   Members   -  Reputation: 150

Like
0Likes
Like

Posted 19 March 2014 - 02:40 PM

Other than to beware of concurrent edits (e.g. network thread adding to the queue while the processing thread is removing from the queue) yes, that can work quite well. The network is just a data transfer medium. You are right to want to keep it simple. Write data, read data, that's it.

 

The interesting part is what you do with that data inside the game.

 

Well, I'm locking the queues while I'm reading or writing to them. The current mechanism tries to;

 

Network thread --

Lock incoming queue and write to it, then unlock

Lock outgoing queue and send packages, then unlock

 

Game thread --

Lock outgoing thread and write to it, then unlock

Lock incoming queue and read from it, then unlock.

 

Just to minimize locking. Probably not perfect, but it works so far and I'm happy.

 

 

 

If any of you have any experience regarding entity-component-systems and how to integrate network to them easily while still keeping them flexible/extendable, the whole point of a entity-component-system, then please drop by in my other thread.


Edited by phayer, 19 March 2014 - 02:41 PM.


#11 hplus0603   Moderators   -  Reputation: 5508

Like
0Likes
Like

Posted 20 March 2014 - 02:39 PM

I disagree. Suppose your enemy is using a spell on you or some other action. You want to get that packet ASAP.


Most actual shipped network games (and their programmers) disagree with you.

Instead, make your gameplay and simulation absorb this time (just like it already has to absorb transmission latency.) Typically, through wind-up animations that are longer on the sending side than the receiving side.

Sure, you're free to schedule your own packets any way you want, and you can probably make a fun game that way, too. It's just that experience says that it's probably not a good return on investment/cost to build a system that sends some kinds of events out-of-line.
enum Bool { True, False, FileNotFound };

#12 spinningcube   Members   -  Reputation: 90

Like
0Likes
Like

Posted 20 March 2014 - 04:37 PM

 

Other than to beware of concurrent edits (e.g. network thread adding to the queue while the processing thread is removing from the queue) yes, that can work quite well. The network is just a data transfer medium. You are right to want to keep it simple. Write data, read data, that's it.

 

The interesting part is what you do with that data inside the game.

 

Well, I'm locking the queues while I'm reading or writing to them. The current mechanism tries to;

 

Network thread --

Lock incoming queue and write to it, then unlock

Lock outgoing queue and send packages, then unlock

 

Game thread --

Lock outgoing thread and write to it, then unlock

Lock incoming queue and read from it, then unlock.

 

Just to minimize locking. Probably not perfect, but it works so far and I'm happy.

 

 

 

If any of you have any experience regarding entity-component-systems and how to integrate network to them easily while still keeping them flexible/extendable, the whole point of a entity-component-system, then please drop by in my other thread.

 

 

Looks good to me!

 

I have a full message/scheduling environment (called Flow) built around lazy type evaluation and the like a bit like AgentC describes in his answer to the other thread.

 

You can build one too if you like, no big deal to it. Also like that there is no distinction to networked or non-networked messages as all individual variables come with full type info (this means that yes bit compressed flags and small variables can take a lot of space - in which case you send a special binary field and compress them all into it) and if you as receiver cannot handle it you just don't. Simple. The approach of creating elaborate class hierarchies, special message container structures and the like is antiquated and quaint (in my opinion), but it is used by industry giants such as google, so it must be good for something :-)


Edited by jbadams, 15 April 2014 - 02:16 AM.
Restored post contents from history.





Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS