Sign in to follow this  

Mobile clients - performance suggestions?

This topic is 2044 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I don't really like the performance I'm getting when running my mobile client against my game server. Any [i]general[/i] suggestions to improve performance (I'm using TCP) aside from setting TCP_NODELAY?

What sort of roundtrip time should I expect as a minimum for reasonably small packets?

There are usually not much data sent, but when the player quickly navigates on the map, they might be sending several move actions a second. Without TCP_NODELAY, I obviously get problems, with lots of commands bundled together, but even without Nagle I occasionally get the results from the server in bursts. (This might be from some unrelated bug, but I don't see it running in the simulator)

Share this post


Link to post
Share on other sites
How much data are you sending, how frequently, and over what kind of connection? For example, perhaps you are sending 20 to 45 byte chunks, and trying to do it 5 times per second, over a 3G phone connection and TCP.

Most network performance issues are a combination of bandwidth and latency, with a bit of error correction thrown in for good measure. Knowing your actual communications pattern is essential to give advice about how to improve it.

Share this post


Link to post
Share on other sites
Frequency depends linearly on player input. If the player takes more actions, data is sent more often. The action where the user has the most chance of sending data packets is when walking on the world map, since there is an input and return data every time this happens. Now I can hide this latency somewhat, but I figure the better I can make it before hiding the latency, the better.

The user inputs something like 40 bytes, receives 90 bytes back for each move. I think the maximum is like 5 times a second, when the user tries to walk as fast as possible.

Share this post


Link to post
Share on other sites
hey, i am currently working on the same stuff. maybe you want to exchange some knowledge? write me a pm if you'd like!

p.s. how are you connecting to the server? what device are you using? in what language and with what libs is the server coded? Can you give us more background information?

Share this post


Link to post
Share on other sites
[quote]with lots of commands bundled together[/quote]

This will happen for all networking, but especially for mobile networking, where noisy environments are a fact of life, and occasional packet drops cause coalescing in re-transmission. Your stream-level protocol needs to deal with figuring out where messages begin/end, and also needs to deal with messages possibly arriving cut in half (first the first half, then the second half.)

Regarding the "burstiness" of connections, that's what you get on TCP over a noisy connection. There are two possible "fixes:"
1) Open more than one TCP connection, and multiplex messages across them (say, round-robin.) This is complex, and only gives you some amount of improvement; a "noise transient" event in the environment may very well end up stalling/re-transmitting all of those connections.
2) Switch to UDP. When packets get there, they get there, and you don't have to wait for earlier packets to get later packets. The link layer may re-transmit and re-order packets; the easiest thing to do is to number each packet and drop any packet that has a lower number than the previously received highest-numbered packet. The draw-back is that, well, packets will be dropped!

Share this post


Link to post
Share on other sites
[quote name='FlyingDutchman' timestamp='1335972389' post='4936780']
p.s. how are you connecting to the server? what device are you using? in what language and with what libs is the server coded? Can you give us more background information?
[/quote]

In order:

- Plain sockets
- iPhone/iPad
- Server in java, POJO NIO
- What else are you interested in?

Share this post


Link to post
Share on other sites
[quote name='hplus0603' timestamp='1335976732' post='4936809']
[quote]with lots of commands bundled together[/quote]
This will happen for all networking, but especially for mobile networking, where noisy environments are a fact of life, and occasional packet drops cause coalescing in re-transmission. Your stream-level protocol needs to deal with figuring out where messages begin/end, and also needs to deal with messages possibly arriving cut in half (first the first half, then the second half.)
[/quote]

I use a simple 2 byte header for the packet length, followed by the packet payload.

[quote]
Regarding the "burstiness" of connections, that's what you get on TCP over a noisy connection. There are two possible "fixes:"
1) Open more than one TCP connection, and multiplex messages across them (say, round-robin.) This is complex, and only gives you some amount of improvement; a "noise transient" event in the environment may very well end up stalling/re-transmitting all of those connections.
[/quote]

That sounds like it would end up rather complex?

[quote]
2) Switch to UDP. When packets get there, they get there, and you don't have to wait for earlier packets to get later packets. The link layer may re-transmit and re-order packets; the easiest thing to do is to number each packet and drop any packet that has a lower number than the previously received highest-numbered packet. The draw-back is that, well, packets will be dropped!
[/quote]

Dropping packets is probably not ok since the current model relies on all updates going through. There is no redundancy in that the same values are repeatedly updated / sent. I read about that neat UDP-trick they used in X-Wing vs. Tie Fighter, where each UDP-packet also contained the data of the previous packet. If both packets disappeared, that was usually indicative of serious connection problems. I suppose one could work around it by enforcing a resend in those cases.

BUT, rewriting server with UDP instead of TCP is nothing I'd like to do unless I'm forced. Most likely I'll simply hide the latency in graphics transitions instead.

There are no other tricks I could employ?

Share this post


Link to post
Share on other sites
[quote name='lerno' timestamp='1335985249' post='4936863']
There are no other tricks I could employ?
[/quote]
Small messages like that tend to get clumped together, as was mentioned.

You could try sending larger messages. Even if the messages are full of empty padding, they are less likely to get queued by various network hardware.

But ultimately you are at the mercy of the network. You cannot control the timeliness of messages.

Share this post


Link to post
Share on other sites
[quote name='frob' timestamp='1335987573' post='4936870']
You could try sending larger messages. Even if the messages are full of empty padding, they are less likely to get queued by various network hardware.

[/quote]

Personally, I'd like to see some persuasive data about this before I went to such drastic measures. I don't believe this would help, and on crowded networks (WiFi, 4G, etc) using more bandwidth *will* hurt overall latency.

Share this post


Link to post
Share on other sites
[quote name='frob' timestamp='1335987573' post='4936870']
Small messages like that tend to get clumped together, as was mentioned.

You could try sending larger messages. Even if the messages are full of empty padding, they are less likely to get queued by various network hardware.

But ultimately you are at the mercy of the network. You cannot control the timeliness of messages.
[/quote]

Actually, I already tried that (sort of). My initial version sent a cascade of separate update packets. Bundling them together for a larger direct write didn't do anything. Not unsurprisingly perhaps, given that I'm using TCP and just pushing packets asynchronously.

Share this post


Link to post
Share on other sites
[quote]I think the maximum is like 5 times a second, when the user tries to walk as fast as possible.[/quote]

So I move forward. Assuming no packets arrive during next second, will I experience jitter or lag, or will I move smoothly?

I really hope that each move doesn't wait for confirmation until it starts showing the movement. That doesn't work even on low latency wired networks.

This is unrelated to quality of network - I should not notice 1 second long absence of communication, underlying engine must deal with it. That is the first problem to solve. On desktop/wired networks, one can go as low as 125ms-400ms (depending on distance to peer), but mobile should really tolerate 1 second, perhaps much more, especially if working on twitch/interactive things.

Ideally, unless doing something CS-like, which isn't really viable directly on unreliable networks, client should ideally tolerate 5 seconds jitter and compensate the way interaction is done.

Mobile networks aren't a single thing. There are dozens or so variations on how data moves, then there's carriers, different wifi issues and whatnot. Edited by Antheus

Share this post


Link to post
Share on other sites
[quote name='Antheus' timestamp='1336127620' post='4937337']
[quote]I think the maximum is like 5 times a second, when the user tries to walk as fast as possible.[/quote]

So I move forward. Assuming no packets arrive during next second, will I experience jitter or lag, or will I move smoothly?
I really hope that each move doesn't wait for confirmation until it starts showing the movement. That doesn't work even on low latency wired networks.
[/quote]

In my particular case:

Since we're walking around on a 2D map and there is fog-of-war [b]and[/b] the client is not trusted with that map data until that fog-of-war is cleared, there is a limitation.

My idea is simply to move the map forward, then fade in the revealed areas. This will give me the whole movement animation to hide any lag. There would be a delay on the reveal until the result comes back though.

In the case the movement actually was refused, the map could drift back to the original position as soon as possible, creating the illusion of bouncing back off some elastic boundary.

Other situations in the game must incorporate similar animations to cover up the lag.

Share this post


Link to post
Share on other sites
[quote]the client is not trusted[/quote]

You're on an iPhone. Just dump the whole map and use different servers for jailbroken clients.

[quote]There would be a delay on the reveal until the result comes back though.[/quote]

Delay is a bad choice here, I personally find this type of map generation very distracting, especially if it's not temporary, such as Minecraft starting up and generating/loading chunks. Jitter is more annoying than lag. If reveal takes consistent 2 seconds it's better than if it's taking 250-2500ms.

Send the map in chunks, keep enough locally to cover 5 seconds or movement if covering for 1-2 seconds of lag.

For the rest, such as units, use the usual observer/LOS method, where only visible/in-range units are updated.

It's precisely the type of design compromise that needs to be made. Pixel-perfect information hiding simply doesn't work well over network. Unless you're Starcraft, but that's a different story altogether and maphacks are detected serverside. Edited by Antheus

Share this post


Link to post
Share on other sites
[quote name='Antheus' timestamp='1336138016' post='4937366']
[quote]the client is not trusted[/quote]
You're on an iPhone. Just dump the whole map and use different servers for jailbroken clients.
[/quote]

Sadly I feel it's not applicable for the game:

Think a multiplayer RPG with strategy elements. You "reveal" the 3x3 grid where you stand. You only see players on the particular tile you're standing on. Player have actions that cause them to alter the map. For the gameplay it is essential that map info does not leak. Clients are not intended to be limited to iPhone/iPad.

During a game session, you typically make around 60 on-map movements. Aside from movement, there are a bunch of other actions.

I feel speculative response is very hard to do, and it's not like it's an action game where you constantly feed updates.

I made a client-server video poker game based on HTTP. When drawing/changing cards, there was a drawing animation as the new cards were dealt, followed by the cards turning (revealing themselves). If there still was a wait after 1 second after the animation, I put up a "Waiting for response..."-spinner. Even with each request sending a new connect, I only ever saw that spinner when having a really poor connection, and then only occasionally.

Besides, does it need to feel jittery? Always animated gliding movement forward (same pace regardless of response), only the delay in the soft fade-in and update of present players would reveal connection problems.

Share this post


Link to post
Share on other sites
[quote]During a game session, you typically make around 60 on-map movements.[/quote][quote]I think the maximum is like 5 times a second, when the user tries to walk as fast as possible.[/quote]

Is a game session 12 seconds long?
---

As for lag and jitter, here's [url="http://www.kongregate.com/games/EdgebeeStudios/sorcery-quest"]an example[/url]. I remember it only because on-move lag was so annoying I consider it to be broken. it's done for same reason, but it ends up terribly distracting.

Maybe it's not a problem, considering it's a typical pay-for-action type of game, but for me it's simply broken.


Is smooth experience worth it? It's rarely mentioned and 10/10 managers agree one optimizes last.

Yet it's interesting to see that even in developers circles, smooth interfaces to get comments like: "it's so fast" or "it's so smooth". IMHO, it plays a factor. Smoothness was one of types of comments on the feeling of iPad, even though there is no real hardware behind it and no magic, just ensuring response rates are faster than in alternatives. And for same reason, Java was always called slow, even when it wasn't,

To me, it matters. Whether it's worth spending effort on that is another question. Edited by Antheus

Share this post


Link to post
Share on other sites
[quote name='Antheus' timestamp='1336143240' post='4937390']
[quote]During a game session, you typically make around 60 on-map movements.[/quote][quote]I think the maximum is like 5 times a second, when the user tries to walk as fast as possible.[/quote]

Is a game session 12 seconds long?
[/quote]

Like your example, movement is something done occasionally, in bursts. A game session will probably tick in at less than 5 minutes though.

[quote]As for lag and jitter, here's [url="http://www.kongregate.com/games/EdgebeeStudios/sorcery-quest"]an example[/url]. I remember it only because on-move lag was so annoying I consider it to be broken. it's done for same reason, but it ends up terribly distracting.[/quote][list=1]
[*]There are two reasons your example might feel slow: The movement uses ease in-out, which is slow in the beginning and end. This is nice for animations that are supposed to feel "gentle", but gives a feeling of sluggishness.
[*]I also strongly suspect that the game performs a new connect every time you move. I.e. you're seeing the latency of TCP connect + request.
[/list]

I believe simply patching two things would change the your of that game: move the animation to linear or possibly ease-out and send requests on an established socket.

[quote]
To me, it matters. Whether it's worth spending effort on that is another question.
[/quote]

I too believe it is important.

Share this post


Link to post
Share on other sites
[quote name='hplus0603' timestamp='1335976732' post='4936809']
2) Switch to UDP. When packets get there, they get there, and you don't have to wait for earlier packets to get later packets. The link layer may re-transmit and re-order packets; the easiest thing to do is to number each packet and drop any packet that has a lower number than the previously received highest-numbered packet. The draw-back is that, well, packets will be dropped!
[/quote]

I did a little experimenting, setting up a rudimentary UDP and TCP echo servers. I then used my iPhone to ping those servers.

Setup:

(Internet) - ADSL - Wireless Router - Computer (connected using ethernet to the router)
iPhone with WiFi/3G sending a single 5 byte packet and measuring the time it takes to come back.
TCP_NODELAY on both server and client.

- With iPhone on the wireless router, UDP/TCP both gives me a roundtrip of 50 ms. So that's just over WiFi directly to my computer.
- Second test was with 3G. The wireless router as a public IP which is forwarded to my computer, again identical results with UDP/TCP - both running between 200 and 500 ms.

(Testing the same thing in the simulator on the same computer gives 2-3 ms, so there shouldn't be anything wrong with the test code)

Why is UDP giving me the same results as TCP here?

Share this post


Link to post
Share on other sites
[quote]Why is UDP giving me the same results as TCP here?[/quote]

Data travels at same speed, so regardless of protocol, it will take the same time.

If there is packet loss, then TCP will try to recover lost packets using a standardized algorithm. This algorithm requires that no data may be sent to application, regardless of whether it has arrived or not, until previous missing packets are recovered or connection is closed.

UDP leaves recovery implementation to you, always delivering individual packets as they arrive (or don't).

If there is no packet loss and route is not saturated, both protocols behave effectively the same and achieve same ping times.


Apparently, in your test, no packets were lost, at least not during the test.


UDP is not faster (ignoring a few bytes of less overhead). They differ in error handling and throttling due to route saturation. UDP may be used to implement an algorithm which handles those two cases in a way that is better suited to the needs of application. Edited by Antheus

Share this post


Link to post
Share on other sites
[quote]
[quote]Why is UDP giving me the same results as TCP here?[/quote]
Data travels at same speed, so regardless of protocol, it will take the same time.
[/quote]

So TCP is not hampered by its Ack, esp since I used TCP_NODELAY?

But then why did hplus0603 suggest UDP?

[quote]If there is no packet loss and route is not saturated, both protocols behave effectively the same and achieve same ping times.
Apparently, in your test, no packets were lost, at least not during the test.
[/quote]

Well actually, I did have one case of aberrant TCP echo (1000 ms) and one occasion where the UDP packet was lost...

Share this post


Link to post
Share on other sites
In TCP, once you drop a packet, any newer, un-dropped packets, will be delayed in the kernel, not delivered to you, until the dropped packet has been detected and re-sent. This causes burstiness.

In UDP, any packet that makes it through, gets delivered, and any packet that doesn't, doesn't.

The "speed" is the same, but the "latency" or "jitter" for game use cases that can stand dropped packets is higher for TCP and UDP.

Share this post


Link to post
Share on other sites
[quote name='hplus0603' timestamp='1336435574' post='4938226']
In TCP, once you drop a packet, any newer, un-dropped packets, will be delayed in the kernel, not delivered to you, until the dropped packet has been detected and re-sent. This causes burstiness.

In UDP, any packet that makes it through, gets delivered, and any packet that doesn't, doesn't.

The "speed" is the same, but the "latency" or "jitter" for game use cases that can stand dropped packets is higher for TCP and UDP.
[/quote]

Ok, that makes sense. There isn't much use for me using UDP then, as in my game the player really needs the response from the last command. There is no good way of predicting the result of an action even if I hadn't the need for information hiding.

You spoke of multiple connections. What about opening two connections, sending identical data across with message numbering? Server/client discards the messages that comes last. Would that help any in real world situations?

In one iPhone game I wrote, I'd have a http request for every time the player did something. I then used a message numbering scheme, and a timeout of about 2 seconds. If the request didn't complete in 2s I'd immediately send the same request again, with the same message number. I did this up to 3 times.

That scheme worked well for avoiding cases when connect would randomly stall on the phone - usually things would work on the second try.

Share this post


Link to post
Share on other sites
Multiple connections *may* help in some real-world scenarios. Specifically, if just one random packet gets dropped, you will still keep on running uninterrupted. However, a lot of problems are in the form of "hiccups" where *all* traffic gets blocked for some amount of time -- this would block both connections at the same time. Also, the cost of this work-around is high, because you double the amount of bandwidth used.

Share this post


Link to post
Share on other sites

This topic is 2044 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this