Select Session Coverage

Designing with Cheaters in Mind

This isn't about how to design your game to prevent cheating. You can't do that. This is about designing your game to mitigate cheating. It's also not designing in terms of gameplay, but in terms of how your game system functions - how all the pieces interact. That's where a lot of player cheating will stem from: finding flaws in the system.

What is cheating? It's interacting with a game to gain an advantage that others cannot. These run the gamut from simply not following the rules of the game (don't create multiple accounts), running exploits (scripts), or abusing design flaws (a certain armor piece remains in affect even when removed). Why should you work so hard to counteract cheating? Because it's simply not fair and will cause players to not want to play your game, and it's just bad practice to leave harmful system effects unfixed that players could use to cheat or potentially even use to hack into your game or other people's machines.

While the design of the game can be the first place to stop cheating from occurring, it won't solve everything. Programmers can also fix issues however they can also create their own as well. Customer Service will see the brunt of any affects cheating has on players, so they're your front line of defense, but Community Managers are your early warning system, as they will be able to spot trends in the community that could lead to the discovery of players covertly cheating.

Why do players cheat? Dr. Richard Bartle has 4 basic types of cheaters:

• Achievers - these people are looking for quick recognition. They want to be on the top of the heap as fast as possible and stay there
• Explorers - not always cheaters per se, they do exploit glitches to get to areas of the world they're not supposed to be
• Socializers - usually looking for payback in some form or another for being socially rejected
• Killers - these players want to dominate others, and usually exploit cheats that let them kill swiftly, or widely

The best time to catch possible cheats is before game launch, preferably before beta testing. Just as with any other change you would make during a project, the sooner you do it he less it will cost. You also don't have to worry about any backwards compatibility or masses of players to piss off by taking some part of the game offline to fix it. However it's easy to miss cheats during this time because 1) developers who playtest don't usually try to cheat 2) you don't have a lot of people playtesting the game and 3) some play testers may actually try to hide a cheat until game launch when they can exploit it. Once a game is live you have thousands of players - the numbers will always be against you.

Catching cheats when the game is launched requires special handling. You have to fix the problem quickly, but not so quickly that you push out a hotfix that only makes the problem worse, or fixes that cheat and introduces a new vulnerability. Clear communication between the Community Manager and the community is key to letting players know action is being taken but time is also being taken to ensure the fix is fully effective. Make sure your Customer Support knows the status of the fix as well so they can inform players calling in for support if you've taken part of the game temporarily offline. Programmers and designers need to be talking to ensure that any fix to the game does not upset the game balance or game play. The whole team has to be mobilized to deal with this.

Detecting cheats quickly can be achieved by a number of means. One obvious way is to have an efficient Customer Support department and Community Manager, so that any complaints from players who have witnessed cheating are handled promptly. A better way however is to actively monitor the community and game to try and discover improper behavior before it becomes a major problem. This can be done with the proper community management tools (flagging system, user activity tracking, etc) and proper game metrics (server logs, client logs, player stat tracking, etc). Study previous games to find out how players cheated to get an idea of what to look for in the data. If you detect a cheat, don't always swoop in right away to fix it. It's one thing to discover a cheat, it's another to actually catch a person(s) cheating.

Top ten lessons:

1. Listen to players about possible cheats - then get specific info from your team and metrics system to confirm suspicions
2. Use mod value for temporary changes - don't modify the base value of an item or game setting, instead replace it with a temporary value so you can switch back easily if things go bad
3. Make on the fly changes easy - build your game to allow for quick and easy changes so you can react to cheats faster
4. Don't put test items on a live server - oops? It happens. Why would you have a one-shot kill item anyways?
5. Don't ignore social engineering - people will believe anything, someone will let a cheat or even an imagined one get to their head
6. Don't let your ego get control - people are out there that are smarter than you. Accept it and accept the fact that they will find ways around your system
7. Don't be afraid to punish cheaters - legit players will start to lose faith if they see repeat offenders only getting a light rap on the hand, on the other hand players will respect the game rules more if the law is laid down. Ex: zeroing of Gamer Points
8. Competition (PvP) makes things worse - be ready for more problems if you have PvP in your games - players hate unfair advantages and they come about often when people cheat
9. Be realistic about cheats - people will find cheats, be ready for it
10. Just keep the game fun - don't punish for the sake of punishment, stop cheaters to make the game fun for others

Pathfinding Solutions for Large Online Worlds

I'll admit this is something I never really thought about myself until now - the fact that in an online game, NPCs have a lot of terrain to navigate over. And they're only getting larger and larger as technology progresses. The usual pathfinding methods don't work as well as they do in single player games, mainly because of the fact that MMO games are mainly server-based and share a lot of resources. The algorithms also don't scale well to the large open spaces in MMOs versus the small, tight FPS levels and perform slowly. MMOs are also dynamic worlds hosting thousands of players, so not only do you have to account for changing environments but thousands of pathing requests per second. And it all has to be extremely stable, lest you bring down part or all of the server and piss off thousands of paying customers.

A typical solution is to generate nav meshes, but John has surprisingly found that a more common word is "create". He was appalled to learn that the majority of MMO nav meshes were done by hand in an editor. This is a bad idea for the aforementioned reason of changing environments. MMOs are not static, and neither should the nav mesh be. These meshes are a layer visible to NPCs that help to guide them along paths and areas were walking, swimming, jumping or flying is allowed. Edges connect to nodes, and characters are directed along the edges from node to node to create a path. It's an expert system that is fully aware of the lay of the land so it also describes the connectivity between nodes in the mesh, for example whether an NPC can jump from one node to another. Interpolating the resulting path can smooth it out so the character's walk looks natural. The nav mesh can also be used to store meta data, such as where NPCs spawn, patrol areas, custom regions and programmable triggers. You also have to make sure to account for various-sized NPCs, either by storing such data in the mesh or creating additional mesh layers. To keep required data to a minimum John recommends classifying NPCs as either small, medium or large. Latency must also be accounted for, as in the time it takes for a path to be created, the character may have moved beyond the starting node, causing it to turn around, walk back to the start point and then turn around again to continue on the path. This obviously looks stupid so allow characters to blend the difference.

John went over several middleware, after admitting how useless he found it to try and roll his own system.

• Kynapse: Didn't have an API. At all. This was very crippling to its use, and also had long computational times
• Havok AI: John found this middleware interesting but was unable to share any data on it. He recommends a look
• PathEngine: This middleware also returned long times in computing paths

John's other reference was the AI Wisdom 4 article on auto generation of navigation meshes.

What you already know about building secure software...

Dave Weinstein is good at his job, and that job is to be paranoid. Of everything. Well okay, I'm sure he doesn't think the coffee machine in is office is planning worldwide domination, and perhaps he's not constantly worried about his house being broken into. But he is on the constant lookout for hackers trying to worm their way into whatever system he is tasked to make sure is secure - and that means not trusting a single bit of data that system receives from a remote client. Likewise, he expects that whatever remote server he connects to doesn't trust a single bit of data that he sends it.

Sounds like a quandary? If both parties can't trust each other, how do they communicate? Very, very carefully.

Hackers aren't motivated by what drives normal people to play games. They simply want to gain some level of control over your system and use it for their own devious means. This sole purpose will drive them to formulate all sorts of attacks, and you can bet that you won't think of them all. So if you can't prevent someone from breaking into your server or hijacking your client then the best you can do is make it as difficult as possible for them, and hope they simply give up and move on to some other, more vulnerable game. Of course, there's also the mentality among hackers to be the first one to beat the system, and that if a suitable challenge presents itself they will work tirelessly at figuring out a way to solve it. That's how things like Blu-ray encryption get broken.

So okay, either way you can't really win. Build that bunker!!

The problem is that, while many people do mistrust data in online games, they usually don't distrust enough. The first thing that security people always look for is corrupt packet data. Older games that performed a lot of client-side calculations fell victim to this. Air Warrior, which ran all its flight models client-side and did not validate on the server, quickly saw players flying higher and faster than they were supposed to be able to. Other client-side data that can be manipulated if it's available to a client can be position, class/level info, authoritative combat, etc. Note that this is all packet data that were looking at here, but actually the entire packet should be considered a security risk, as hackers can modify the size, format and sender (UDP injection - which affects many online games that take advantage of UDP vs. TCP) to break through to your system. This is where you should apply a technique known as fuzzing to see if your system is susceptible to these forms of attacks. You should also review features that can be unsafe by design such as a tech-support ability to run commands on a user's machine or an auto-update feature that could be hijacked to send malicious updates to clients.

So you shouldn't trust you clients. But your clients can trust your servers right? After all, they're usually directly under your control. The client connects to the server and sends a verification code. The server goes okay! and you're all good right? Uhm, are you sure you're actually connecting to the game server? Internet connections are generally done by hostname, and just asking for server.game.publisher.com doesn't guarantee you a connection with the actual game server. First, there's DNS poisoning. Then there's also ARP impersonation on a local network. How to hackers get on your local network? Well it turns out that wireless routers in the same area can have the same SSID and password. Are you sure you're connecting to your router when you log on to the internet? Connecting to a rogue server can allow inappropriate content to be streamed into your game, or even allow the game to be patched with malicious code as Evilgrade demonstrated. So clients can't trust servers either, and mutual authentication is required.

Another threat is posed by scripting systems. Lots of games have them to allow their users to create new scenarios or objects, and lots of hackers love to exploit them to run programs that give them and advantage or control over some part of the game. Usually these problems manifest themselves after the game has been out in the wild for a while, but a lot of vulnerabilities can be found via an audit by a security professional. These are people whose sole purpose is to pour over your code and try to break it. They can be well worth the additional cost to ensure that your scripting system (and other systems) are secure so that your players can get full enjoyment out of extending their game play experience. This should also be something that is actually planned into the development cycle, not a last-minute decision that will cost you tons of money if anything needs to be fixed/overhauled/cut.

Finally, don't forget the data that you store on people's machines. Media formats are complex, large and in binary - hackers like this. Anything stored as data can be interpreted as code. While games usually include custom complex formats with accompanying custom parsers in the game to read them, humans have a distressing tendency to write vulnerable parsers - especially if they control both the tools that produce the content and products that consume the format. People always think it’s good enough for them - but there's always someone better than you. Some examples include replacing camouflage colors in a texture with high-contrast colors - now everything is extremely visible. Or increasing the volume of footsteps so the player can't be snuck up on. Or changing model data to alter the hitbox. This is where fuzzing comes back into play again. Fuzz your parsers, and your signature validation code for the client/server.

The hackers are out there, and they're gunning for you because you're a new system that hasn't been tested yet and they all want first dibs. Fuzz all exposed data surfaces, any user provided content and network traffic. If you can't hire a security expert to audit your code, it may be best to simply not expose code to the player through a scripting language. Security experts like to say "think like an attacker" but that doesn't just apply to how an attacker will attack your system, but why.

Building Like You Play: The Mechanics of a 100% Virtual Development Team