Your most valuable debugging techniques for Networked games?

Started by
15 comments, last by Kylotan 9 years, 10 months ago

Make sure that your game launching code lets you create multiplayer games with just one player in debug mode, make it so that it uses as much of the regular multiplayer logic as possible.

Put systems in place to allow creation of AI driven clients (ideally running in a console window on the same machine, so you're not wasting time with multiple machines)

Advertisement

Logging to a file and simultaneously logging to a seperate visible logging window.

Add a small library of logging tools to make it very easy to add logging message calls (including major/minor indicators in the logs to make important messages stand out, and useful things like imbedded timestamps, dumping of variable values, socket state blocks, etc..)

Having a way to subset messaging to turn off logging of 'priority' sets to lower that mess of log data to look thru (sometimes you want to turn off the loging for ordinary stuff , but once something goes wrong you want it all back again and adding/removing logging message calls over and over is wasteful)

Dont forget to flush you logger (file) every message (mine usually opens/closes the file every message) and with threading usually the case adding the protects for the read/write operations...

--------------------------------------------[size="1"]Ratings are Opinion, not Fact

There has been some great tips, I think I will need to upgrade my logging classes quite a lot. I particularly like the idea wodinoneeye had about having different priorities of logging. I'm thinking of making it possible to change this in real time as well so although there will still be one large log file saved, I will also have another log file that will only apply for certain types of log messages.

- Make sure you also do some unit testing. smile.png

More stuff, maybe :

- Internet conditions simulation. Either basic, for a quick test, or pro-level stuff, with dedicated hardware. Will not replace real-world conditions but I find it always useful.

- Since you're dealing with binary data (most likely), some form of checksum / type-checking / range-checking verification. This mostly applies to debugging things like serialisation / deserialisation and marshaling.

- Some form of binary -> readable text conversion on packets. Usually packets are tightly packed binary streams and virtually unreadable. So, some form of analyzing packet content can be useful if you run into trouble with your packets turning up as garbage.

- Recording packets, game states, inputs, and replay features. Pretty high level, and a lot of work in itself, so you may want to skip that.

Checksum and replay especially useful if you have the intention of running a deterministic lock-step engine.

At a game level :

- A basic versioning system. If your packet / message format isn't backward compatible, you'll end up with garbage data.

- Dependencies check. Making sure whoever connects to your server has all the custom content it needs to being able to play the game. Either the right version, the map data, ect...

- A replay system can also help you catching strange game exploits. And if your game supports replays, then why not smile.png

Everything is better with Metal.

Unit testing is the only sane way to handle this. While integration testing is popular in the industry, there is some fairly compelling data out there that shows that the effectiveness of integration testing goes down exponentially the more complex the system gets as a whole. My own experience tends to concur.

Especially for networked applications, I am a strong proponent of using a lot of mocking and stubbing. You want to keep your tests isolated to what you want to test, not all the other stuff.

Unit testing is also a good incentive to write better abstractions. Good abstractions are just easier to test, and you will find yourself just writing better overall code when you start unit testing.

A follow up from my post.
I've been changing a lot of serialiser code and I have my unit tests run whenever T import my library (for debugging purposes, of course). It's caught bugs that I'd have to labour over otherwise!

I may as well chime in with some late observations.

  • sending the wrong packet data to the server.
  • the server unpackaging this data incorrectly.

As already mentioned, these 2 are perfect candidates for unit testing. Every single part of your serialisation and deserialisation code should be trivial to write tests for and therefore you can be in a position where you are strongly confident that there are no bugs here.

Another idea I like to implement in debug builds is to immediately unpack any packet I'm about to send and verify that the unpacked version matches the packed version, using an assert. This catches several types of serialisation bug before the data even hits the wire.

  • the server sending the wrong data to the clients.

Obviously this would be a general logic error, but one way to reduce this problem is to limit the variety of data that can be sent. Some games create hundreds of different packets, one for every piece of functionality. If you can reduce these disparate messages into a much smaller set, then there is less scope for error. My MMO code has no more than roughly 10 distinct message types, and although each of those has some subtypes, they all pass through the same few basic functions and are easy to check for errors.

  • the server applying the data in the wrong way.

Again, this is a general logic error, no different from doing the wrong thing in any code. But if you have reduced the number of distinct messages then it would seem harder to do that.

  • the clients unpackaging the data incorrectly.
  • the clients applying the data in the wrong way.

Ideally your clients and servers use the same code, so if you fix the problem at one end, you'll fix it at both. But if you can't share the same code - and I know what this is like, having worked on an MMO with a Python server but clients written in C# and C++ - just try to keep the code as similar as possible and unit test as much as you can.

This topic is closed to new replies.

Advertisement