Your most valuable debugging techniques for Networked games?

Networking and Multiplayer Programming

Started by JordanBonser April 23, 2014 09:26 AM

15 comments, last by Kylotan 9 years, 10 months ago

839

Author

April 23, 2014 09:26 AM

I have recently started adding networking to a game I have been making but the development of this in comparison to a single player game seems extremely slow. This is mostly due to the complexity of debugging issues that occur. As a general rule the problem can be:

sending the wrong packet data to the server.
the server unpackaging this data incorrectly.
the server sending the wrong data to the clients.
the server applying the data in the wrong way.
the clients unpackaging the data incorrectly.
the clients applying the data in the wrong way.

So this can get quite complex and the main ways I have been debugging so far is through either trying to debug normally in the IDE. Outputting a lot of Logging information, which can be tedious. Outputting some debug information to the screen which is often fairly limited but can show problems in real time.

I'm just wondering if anyone else has any other better ideas or any network debugging tips that have really helped them?

Ashaman73

13,718

April 23, 2014 09:50 AM

I would focus on the log-file. How about writing a log-file analyser,which looks out for certain errors, check protocol messages, compares protocol server msg with client msg etc. It could be a valuable investment.

Ashaman

Gnoblins: Website - Facebook - Twitter - Youtube - Steam Greenlit - IndieDB - Gamedev Log

JordanBonser

839

Author

April 23, 2014 10:40 AM

I would focus on the log-file. How about writing a log-file analyser,which looks out for certain errors, check protocol messages, compares protocol server msg with client msg etc. It could be a valuable investment.

Ahh yeah, I never really thought of making it that descriptive. So far I have literally just output certain values, like the values of a matrix and checked them by eye when I have needed to. So to expand a bit, you would recommend outputting the values of the packets before sending and when recieved to a log file.Then create a small program to scan the file and check the values of the packets, throwing an error if any differentiate?

Ashaman73

13,718

April 23, 2014 12:11 PM

Then create a small program to scan the file and check the values of the packets, throwing an error if any differentiate?

Yes, this is one option. More important are protocol msg order, eg


Client 1 request game slot
Client 2 request game slot
Client 1 assign slot 3
Client 1 sends user data
Client 2 assign slot 4
Client 2 sends user data
Client 1 accepted user
Client 2 denied user
Client 1 request level data
Client 1 sends join signal
Client 2 sends join signal
<crash....>

One option would be a filter like this


Client 1 msg:
- request game slot
- assign slot 3
- sends user data
- accepted user
- request level data
- sends join signal

An second option would be analysing the protocol:


Client 2 sends join signal after being denied !!

Which could be the cause of the crash.

Ashaman

Gnoblins: Website - Facebook - Twitter - Youtube - Steam Greenlit - IndieDB - Gamedev Log

nfactorial

735

April 23, 2014 12:13 PM

You should have unit tests for all the issues you list. That would eliminate packing/unpacking/applying from your list immediately at the very least.

JordanBonser

839

Author

April 23, 2014 12:36 PM

Ahh yes that does seem to be a much better way of interpreting the data and finding the cause of crashes, thanks a lot :) Yeah I should definitely write unit tests for all my packet classes, another great tip!

Is it worthwhile do you think to have some sort of GUI for checking the state of objects in the scene? maybe like a console with special commands?

Angus Hollands

869

April 23, 2014 12:58 PM

Unit tests are a god send!

I've never been exposed to their necessity in a production environment, but now I don't think that I will need that experience to encourage me to use them. If you can't write a unit test for it, it probably needs refactoring!

Most network errors are caused by misreading data from the byte stream (off by-n errors).I always check the data coming in. However, recently I had an error in some reading code which caused an off by one error. The best thing to do in those circumstances (my error only arose after data was already sent) is to break the simulation into three different states and compare the transition between them (whenever my variable was set to None, it remained considered as a None value (Which isn't included in the byte stream) hence we forgot to read an extra byte from later packets! I had to break it down to fix it.

Having an inspector in game is really useful, (assuming you're a runtime interpreted language) even a GUI with a command input dialogue is all you'd need (as long as you can prevent the game state from progressing)

oliii

2,202

April 23, 2014 01:26 PM

logging everything.

Everything is better with Metal.

Glass_Knife

8,637

April 23, 2014 01:45 PM

1. Rolling log files that keep a certain amount of data (10 Mb) or so you always have enough to go back and find the problem. If it would help, you can log data in XML or JSON or some kind of parsable format so you can generate HTML views of the data or sort through it with code. This can help in with finding problems you didn't know were there.

2. Wireshark - Some kind of network protocol analyzer can really help with weird bugs. You won't need it often, but sometimes this is the only way to figure out what is happening.

3. Testing - there should be a clear separation between the networking code and the rest of the system. I always have a Network interface or something like that. You can create a mock network for testing that simulates all the weird things that can happen: packets out of order, lost data, etc. If you assume that the network code works, you can test all the code that uses the network to verify it behaves as expected. It isn't possible to test all the network functionality, but if you test that your code gives the correct stuff to the network class, and anything taken from the network class is handled correctly, then there is a lot less that can go wrong. And when you find that the server behaves in a weird way, you can test that your classes respond correctly to the new buggy behavior you've discovered.

I think, therefore I am. I think? - "George Carlin"
My Website: Indie Game Programming

My Twitter: https://twitter.com/indieprogram

My Book: http://amzn.com/1305076532

hplus0603

11,916

April 23, 2014 03:52 PM

but the development of this in comparison to a single player game seems extremely slow

Yes, it is, because a distributed system is a lot more complex than a self-contained system!

And it will keep getting slower, because you haven't yet brought up issues where "player A sees something different from player B" or "this problem only happens when that particular packet doesn't make it to some of the players" or "timing."

I agree with the other posters -- keeping a log of all packets sent and received since start-up is a good way to debug, and unless your server stays up for days on end, you should have ample disk space to keep the entire log around.

Also, once you have the "converting packets to structures and back" marshaling part of networking down in a re-usable library, that won't really be a problem anymore. That's when the real challenge of distributed computing starts!

enum Bool { True, False, FileNotFound };

Your most valuable debugging techniques for Networked games?

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Your most valuable debugging techniques for Networked games?

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines