Sign in to follow this  
824learner

Why everyone develop MMORPG in client/server mode?

Recommended Posts

824learner    122
I am new to this, and feel quite curious about it that why most MMORPG are developped in client/server mode, that is using a centralized server? why don't they just use something dicentralized game logic, that distribute the processing/store/... load on customers? what's the problems they meet? security? any help is appreciated. 824learner

Share this post


Link to post
Share on other sites
824learner    122
thanks a lot!!!

so technically, security (anti-cheating) and easy network structure detect/update are the two main challenges p2p mmorpg are facing?

I still have one question, for I suppose players won't need to change their network connections very frequently, then can't the algorithm used in bittorrent be used in p2p mmorpg? why are these two different on a technical level? (besides the security problem)

any help will be appreciated,
thanks,
824learner

Share this post


Link to post
Share on other sites
Bob Janova    769
I think the main problem with any sort of P2P architecture is that it would make it much easier to cheat. If the checking code is being run in the server, you can't affect it; if it's on your machine you can.

The other thing is that you could get large latencies if your peer was a long way away, or if they couldn't upload data fast. For downloading stuff this doesn't matter, but for games consistently low latency is fairly essential.

Share this post


Link to post
Share on other sites
CTar    1134
Imagine a MMORPG with just 1000 players on a server, each player will have to send its data to 999 other players, and it will have to recieve data from 999 other players. So imagine if we had to send 10 KB/sec in a client/server architecture, in a P2P architecture we would have to send ~10 MB/sec. Also imagine if the client could determine which data it would give itself, it could say it had the best sword in the game, and there would be no way to say that it was wrong.

Share this post


Link to post
Share on other sites
Ezbez    1164
And, correct me if I'm wrong, BitTorrent requires multiple computers to have the same file for a really fast download. This means that when you send, say, your characters position to all the other computers, only you would have that information initially, meaning that there would be no increase in speed. You might be able to get marginalaly better performance if once a computer recieved your position, it would also send it out to other computers that needed it. This would be a very marginal benefit and might not even work.

Edit: After reading CTar's post, heres another thought:

On a non-p2p MMO, the server would hold all the players positions. It would only need to send out the player's information to other players near-by. This would, obviously, be more efficient than sending it to everyone.

Now imagine a p2p MMO. Each client holds its own positions. In order to know whether or not it needs to retrieve/send data to another player, it would be necessary for it to already know the position of that player. This means that every position must be sent to every computer every frame. The standard MMO will thus have better performance than the p2p one.

Share this post


Link to post
Share on other sites
cherryhouse    100
Quote:
Original post by Ezbez
And, correct me if I'm wrong, BitTorrent requires multiple computers to have the same file for a really fast download. This means that when you send, say, your characters position to all the other computers, only you would have that information initially, meaning that there would be no increase in speed. You might be able to get marginalaly better performance if once a computer recieved your position, it would also send it out to other computers that needed it. This would be a very marginal benefit and might not even work.


It could possibly have an increase in performance actually, especially with the "zones" in a server. I'll try to explain as best I can my view on how this would work.

You have an initial server hosting 5 areas of a world. Each of those areas in the world is, in reality, their own server. So you are running 5 servers distributing the player locations. Now if you have that single server distributing 5 locations to 10000 different players, lets assume there is 2000 players in each of the 5 world areas. Lets divide these areas up now, we will "disect" one area. The main server would send the location of a player to a single user. So now, this user now contains the information as to where player 2 is located. He then starts distributing his knowledge to all the other players and it continues like that in a web pattern. Once a player separates lets say 50 feet from another player and is out of range, he would stop sending that players information to other players because he is not his business anymore. Each client connected to the main server would now be considered their own server while distributing information equally no matter how many players are in that area.

It's a confusing little concept but it could possibly work. This is my idea of how it would work but if anyone has any others let me here them.

By the way, if what I said above does make any sense, this would mean that once you have about 5000 users on the server you would be able to play the game without even having a main server online, meaning players wouldn't be able to be shutdown (besides kick commands), unless you had a master server that watches all the "sub-servers" that begin the main hosting of the world server.

Share this post


Link to post
Share on other sites
Tenjou_Utena    122
Here's a list of reason why a p2p network for any sort of game really won't work. I'm going to be comparing to BitTorrent, since it's the P2P I know the most about, but this is mostly applicable to any P2P.

First off, to understand webbed P2P networks, you really need to know something about Graph Theory (http://en.wikipedia.org/wiki/Graph_theory).

The first really big obsticle in P2P gaming is that there's no real easy way to tell if your graph of clients is complete. I.E. that ever node int eh graph has atleast one route to every other note in the graph. I believe this problem is O(N!) But I'm not sure. What this boils down too is that your topology might change such that there are two distict graphs, and while they both appear healthy, nodes from one cannot communicate to nodes with another. Imagine a Star of David with Nodea at each point. You really have 2 graphs of 3 nodes here, not 1 graph of 6 nodes. This is not a trivial problem, but there are others.

In BitTorrent, when you download the .torrent file, it has a lot of information in it. It tell you Where to get peers, and also what files to expect, their checksums, and sizes. This means that when you start into a BitTorrent session, your client already knows what information it needs. It has X pieces it needs. This means that the vast majority of the traffic incoming into your computer is actually wanted traffic. In Gaming, your client doesn't know what specific information it needs. It needs 'Info about where I am' 'Info about NPCs' but it doesn't know that IRTank is right next to you, and you want to get IRTanks info. So, you can either trust the clients to tell you what's around you (And maybe they don't know); or you can send all traffic anywhere, which would be like you having to pass the traffic for EVERY PEER that wants it in BitTorrent.

BitTorrent doesn't care about lag, in the same sense that gamers do. If you get a bit of a piece 30 seconds later then you thought you would, it doesn't matter much to BT. But in Gaming, that could mean the difference between life and death.

Then we come to the trusted network model. In BT, as I already alluded too, Your client knows what it wants already. So, if it gets bad data (Happens more then you might think) It can disguard it. Your gaming client, on the other hand isn't really going to know bad data, especially if that data is well-formed.

Plus, you can't trust your players not to cheat. Here's a conceptional list of what happens in a game that might be tampered with:

Equipment -- Just look at the original diablo for the problem with this.
Random Rolls -- Does your spell hit? How much damage do you do? Who decides all this? You can't trust the client. You don't want to trust someone else's client.
Positional Information -- You can easily teleport, of the clients are going to trust you on where you are.

That's just what I can think of off the top of my head. Some of this is detectable (Positional... Did they go too fast?) But then, what do you do with that client? Drop them? It's hard to say. Plus then your client is doing the job of checking everyone for positional data.

Plus, It's easy enough to tamper with the network of fringe nodes fairly easily. Tell them bad info, drop them, create artificial lag.

You also don't have any structure for keeping track of what in the world. ANd how do NPCs act? Monster actions are randomized (Or way too easy to deal with). Who controls this?

Last point: Who's going to pay you a monthly fee if you don't run any servers?

Now, you might beable to save some bandwidth by keeping Server/Client for crucial game functionality, but shoving the non-crucial down to a P2P layer. Chat, guestures, skins, appearence, etc. could be handled by a P2P network failry easily, because if this info is wrong, it doesn't effect gameplay too much. But you really haven't saved yourself the server.


Share this post


Link to post
Share on other sites
824learner    122
thanks you guys for so much replies, but I am more confused, I mean, why should a player send his information to all other players, I mean he only need to interact with a few players nearby? By the way, won't the "distributed" component arise from map storage rather than player ... I mean, the map can be stored distributedly, for example, a room store on this player's machine, a town on another player's more stronger machine, then all player inside this town will interact with this player's clinet (also as server)? also just as MSN, or Bittorrent, there will also be a globally master server, that perform some necessary coordinating work, such as p2p establishing, billing, login etc. so namely it just distribute what can be distributed, ? there could be many problems, but there are also pros, such as player can edit their "home map", (like in sims? never played.) I suppose maybe a lot of players might be interested in building a home in game, just as many guys interested in making a legendary character.

for the cheating problem, I really have no idea, (I know quite littel about it, but I am curious why don't they write it into some license agreement, such that who cheat would be considered illegal?)

thanks,
824learner

Share this post


Link to post
Share on other sites
cherryhouse    100
The reason you would have to have players send their data to everyone is for things like cross country interaction. Mail, /whois which displays character location etc, private messages, and more..

Share this post


Link to post
Share on other sites
hplus0603    11348
The main challenge is cheating/security.

Another of the challenges is: if I am far away from you, and thus we're not currently sending information to each other, and I then move close enough to you that we should "discover" each other, then how does your peer and my peer find each other? Doing this entirely peer-to-peer is not yet a solved problem; there's always some seeding server that starts up the process in the architectures I've seen.

A third is that of efficiency: when you use client/server communication, then you send only your data to the server, and you receive a consolidated packet with the N closest entities from the server. Header overhead is small, and upload bandwidth use is small. In P2P, you have to send your data to each of the N nearby peers, which means your upstream is suddenly multiplied by N; further, you'll receive updates from the N closest peers as individual packets, instead of consolidated packets, for much less efficient usage of your network link. This is extra bad because most consumer broadband connections are asymmetric -- there are cable connections with 4.0 Mbit down, and 48 kbit up!

If you really want to look into P2P MMOGs, then I suggest you google for the "VAST" project; it's an academic research project and open source effort on the subject.

Share this post


Link to post
Share on other sites
intrest86    742
Hey everyone. This topic comes up often enough. Suffice it to say, this is a more diffilcult problem than the OP recognizes, but not as difficult as everyone else has been saying. It is not impossible to fix at least some of the problems mentioned here.

Possibly the easiest to fix is "having to send positions to everyone". Instead of considering your network distributed clients, consider them distributed servers. Instead of each "client" holding their own info, they are responsible for some other set of information. For example, one client could be a set of shops, another NPCs, and another player positions within a zone. On top of all this put a server that knows who has what. The server just needs to distribute that info as needed, and then the clients can talk ot each other to actually play.

Security is a more difficult beast, but steps can be taken. Redundant data across clients is one possibility, where the server checks data veracity every once in a while or only checks when clients disagree with each other. By intelligently copying data across clients you make it so that a cheater has to control a large percentage of clients to cheat (at which point you could even instance a new world just for the cheaters).

Of course, every possibility also raises questions of bandwidth. Creating data redunandancy also means increasing the required bandwidth for each client. Doubling redundancy can also mean doubling bandwidth. You also have to deal with the normal connection issues like lag, except with someone elses computer playing the server.

Share this post


Link to post
Share on other sites
cherryhouse    100
Sadly, it isn't as simple as assigning each client a small load of the world(shops, npc's, etc). On the other hand, if you "packed" your server objects, you could assign, let's say, a pack of 5 npc's from the graveyard to 2 clients. This would cause the server to have a backup type system incase one player drops, bringing us to the question: What happens if the 2 clients drop from a power outage in their area and they were assigned the same pack? The npc's in the graveyard would have no interaction with the game world? They would disappear until re-assigned?

Share this post


Link to post
Share on other sites
hplus0603    11348
Before you say that redundancy is a solution, I encourage you to actually try implementing it, and trying to avoid world splits and keeping everything consistent. Let us all know when you've succeeded ;-) It's a really hard problem, that's not yet been solved in this context.

Share this post


Link to post
Share on other sites
uncutno    146
I have also tought about this, and what i found out is that you
have to redesign your game :-)

For example, you could be an iseland in an iseland world,
Iselands dont move, so P2P communication with nearby islands
would be possible, and would benefit your server. Your world could consist of simulation objects, like a bullit from A to B, and then maybe 3 peers could precalculate the result, and then compare the different results?

Slow changes could be a keyword here :-)

The free roaming open world MMO, with high speed action isnt benefitial
to do P2P as i see it.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
There has been talk (in at least one game programing book I saw more than a year ago) about having some server functionality (data processing) farmed out to the client machines. To overcome the reliability and security issues, part of the system would do testing to establish 'trusted peers' and have the data processing be sufficiently mystified and encrypted to make it hard for any cheater to try to glean useful game info (and change it as well).

Batched tasks would be sent from centralized servers out to the processing peers and could change constantly so that it would be hard to intercept related data for any length of time. Certain AI functions could be done (and the higher downlink speed ratio isnt as prohibitive, as usually the results are much smaller(uplink) than the data (world situational data) used as input (downlink).

Mystification measures might include AI scripts/dll code being resent on the fly to run -- possibly patched/recompiled daily to make reverse engineering in real-time nearly impossible (because you dont get the code til its to be run). Data could have any recognizable game identifiers removed/encoded (basicly a bunch of ordinals representing some small part of the map and unknown objects...) and decoded when reinserted into the central servers.

Tasks can easily be reassigned if a client goes down and statistics would be kept to decide which 'peers' can be 'trusted'...

Share this post


Link to post
Share on other sites
Bob Janova    769
Interesting, but I imagine the processing involved to keep track of which tasks are to be sent to which clients, which are in progress and so on would be as complicated as just doing the tasks, unless there's some really complex algorithms being run.

Share this post


Link to post
Share on other sites
hplus0603    11348
Quote:
have the data processing be sufficiently mystified and encrypted


That doesn't actually work, because no matter how encrypted or mystified something is, it can be reverse engineered, and the data snatched out of the memory as the CPU is working on it. A secure or trusted computing system must be built on algorithms, not voodoo.

Regarding farming out "batches of work" to clients: the reason we have servers is that games require low latency responses. Servers can have context for all questions they need to answer, and thus arrive at a low latency answer. If you "farm out" "batches" to peers, without those peers having sufficient context, your latencies will skyrocket, and the game will not be responsive.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
Quote:
Original post by hplus0603
Quote:
have the data processing be sufficiently mystified and encrypted


That doesn't actually work, because no matter how encrypted or mystified something is, it can be reverse engineered, and the data snatched out of the memory as the CPU is working on it. A secure or trusted computing system must be built on algorithms, not voodoo.

Regarding farming out "batches of work" to clients: the reason we have servers is that games require low latency responses. Servers can have context for all questions they need to answer, and thus arrive at a low latency answer. If you "farm out" "batches" to peers, without those peers having sufficient context, your latencies will skyrocket, and the game will not be responsive.



As always its a matter of making it harder (from trivial to time consuming -- especially to destroy recognizeable patterns that act as tags that give clues to related data).
Combine that with having the task program's code mutate (new differently compiled resent every day -- or even more frequently) to make it nearly impossible to reverse engineer fast enough (ie- require it be done manually) before it changes again.

More likely, having the tasks be randomly reassigned frequently will do more to making hacking worthless -- except for the sick people who simply want to disrupt the servers operations or arbitrarily pervert the data for whatever players are being processed.


As someone else mentioned, the distributed processing tasks taken by these 'trusted peers' have to be complicated enough (low transfer overhead to work ratio) such that it woulldnt be cheaper just to execute on the promary servers.

The problem usually is that there often is alot of current situational data which must be initially provided and kept up to date. eample- an A* pathfind on a map requires the map data, which may be quite large (if it was small, the overhead would make running it remotely more wasteful) and map updates (probably including dynamic objects) would be required.

More complicated tasks like running a 'zone' may be too large a task to be done at the same time as a running client interface. Or if they attempt to do it by breaking the world up into many small zones, then the inter zone communications
start becoming prohibitive -- more cross boundry events, more zone->zone transitions that require protocols where internet type lag gets ugly...
Failure would be massive (detrimental to multiple players) in that zone locality until the task could be reassigned -- too risky for a bussiness run game.

Maybe running NPC/monster AI (with a smaller local world representation than the players avatar gets??) -- this would be to the server the equivalent of a second client connection (transmission of map data window/object update events stream, etc...). NPC AI is one of the things that needs significant improvemnet and the computing requirements can be $ubstantial (and the failure path can fallback to simpler default 'crappy AI'(tm) that we have now....



One of the other advantages of such a 'peer' system might also be the communications bandwidth costs can be distributed (versus the usual large pipe into a central server) to go along with the lower cost of fewer company run servers....











Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
Quote:
Original post by hplus0603
Quote:
have the data processing be sufficiently mystified and encrypted


That doesn't actually work, because no matter how encrypted or mystified something is, it can be reverse engineered, and the data snatched out of the memory as the CPU is working on it. A secure or trusted computing system must be built on algorithms, not voodoo.

Regarding farming out "batches of work" to clients: the reason we have servers is that games require low latency responses. Servers can have context for all questions they need to answer, and thus arrive at a low latency answer. If you "farm out" "batches" to peers, without those peers having sufficient context, your latencies will skyrocket, and the game will not be responsive.



Not all tasks require 'twitch' response. Some AI tasks are ongoing and/or can make use of the 'thinking' type delays seen frequerntly in games. Immediate responses like action animation sequencing and battle behaviors can still be done on the primary servers, but higher level behaviors that dont require instant results are possible (some games today use round-robin pathfinding spread over many 'turns'). Another use would be optional optomizations where pathfinding are redone continuoulsy and if a better path is found it would replace the existing one, and if unreliable isnt to disruptive.

There are background tasks like economic simulation and incidental behaviors (guiding those pretty butterflies around the map....) that arent high priority.
Quest generation could be a candidate -- running scripts to do validation/pattern matching/fitting can take quite a bit of time and the results arent usually needed instantly.






Share this post


Link to post
Share on other sites
hplus0603    11348
Quote:
One of the other advantages of such a 'peer' system might also be the communications bandwidth costs can be distributed (versus the usual large pipe into a central server) to go along with the lower cost of fewer company run servers.


In a peer-to-peer system, I need to send my data to N other people, and I need to receive data from N other people, and I need to send data about more than just me, for redundancy, error resiliency, and security auditing.

In a client-server system, I send data to one place, and receive data from one place.

Clearly, the client-server system uses significantly less bandwidth. If the price of bandwidth is a marginal cost (which, in aggregate, it is for society at large), then client/server solutions are actually more economically efficient, and deliver better experiences for cheaper aggregate cost than peer-to-peer.

To view it another way: You would need, say, a 4 megabit peer-to-peer connection to get similar network fidelity as you'd get from a 1 megabit client-server connection. If the client/server costs less than the price difference between those two connection types, it's actually a win, even for the user.

Quote:
Combine that with having the task program's code mutate (new differently compiled resent every day -- or even more frequently) to make it nearly impossible to reverse engineer fast enough


In my opinion, no organization could QA code that mutates several times a day. There are only a finite number of scrambling techniques, after all. The best you can do is change parameters of a known algorithm, such as keys. There are simple attacks that run deltas on diffs (because you need to distribute these changes to all players, right?) and determine what the change is. If you think the user machine can be trusted at all, you probably shouldn't spend your time on trying to frustrate the untrustworthy users; your time is better spent on designing game mechanics that don't benefit from client cheating.

Share this post


Link to post
Share on other sites
middy    133
Take a look at my thesis I have several examples of P2P MMORPG models as well as anti-cheating measures. The thesis deals with RTS games but the model can easily be used for MMORPGs


www.p2pgamenetwork.com


Middy

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
Quote:
Original post by hplus0603
Quote:
One of the other advantages of such a 'peer' system might also be the communications bandwidth costs can be distributed (versus the usual large pipe into a central server) to go along with the lower cost of fewer company run servers.


In a peer-to-peer system, I need to send my data to N other people, and I need to receive data from N other people, and I need to send data about more than just me, for redundancy, error resiliency, and security auditing.

In a client-server system, I send data to one place, and receive data from one place.

Clearly, the client-server system uses significantly less bandwidth. If the price of bandwidth is a marginal cost (which, in aggregate, it is for society at large), then client/server solutions are actually more economically efficient, and deliver better experiences for cheaper aggregate cost than peer-to-peer.

To view it another way: You would need, say, a 4 megabit peer-to-peer connection to get similar network fidelity as you'd get from a 1 megabit client-server connection. If the client/server costs less than the price difference between those two connection types, it's actually a win, even for the user.




Consider MMORPG that have a server farm (seperate sets of Database, Game mechanics, AI, Webserver machines). The arrangement is not really 'all to one place' machinewise and the worlds are broken up into zones/areas/rooms that execute with subsets of all the players in the game (N is a fraction of total players).
The worlds are sometimes broken up into hundreds of zones/areas/rooms thus making the N much smaller (not ignoring that there still can be areas like the 'Banks' and temporary 'event' locations where much higher numbers congregate).

Even with your 4 to 1 ratio, if there are a 100 usable clients to 1 server you still have a whole lot of potential bandwidth that could be put to use (for free). Scale that up to a large MMORPG with more than 10000 active users (1000 trusted peers??) ----- might that not add up to $$$$ saved???

As to "price of bandwidth is a marginal cost", the players may now have cheaper/larger bandwidth, but the games will try to make increased use of that bandwidth and the servers will still require some N multiple of that bandwidth amount -- which means it remains a big $$$$ expense. (already there are games that dynamicly load assets (map segments, textures, music, sound effects) on the fly and if anything, that trend will increase).


Actually I dont see it practical to run the 'trusted peers' as full blown areas/zones/rooms servers, its likely not reliable enough. BUT, secondary uses
as chat servers, voice servers, Asset servers (on future dynamic map/content games), Patch downloaders, NPC AI, Scenario generators, etc... could make use of the communication bandwidth as well as the CPU capabilities of the distributed 'peer' servers.

Share this post


Link to post
Share on other sites
_winterdyne_    530
A trusted peer methodology is really no different to the cluster architecture. Practical differences do apply (a far more unreliable infrastructure), as well as security considerations - actually letting players have a trusted peer is probably a bad idea, without some definite method of ensuring the peer is behaving correctly (which places load on your private 'seed' servers).

Really, it's a business risk - assuming your game is free, and no-one stands to gain anything in particular from hacking your peers, the risk to yourselves and your players is greatly reduced. A robust messaging system at the client end (the client knows what to expect) would allow compromised peers to be reported and removed from the heirarchy of servers.

If you were thinking of actually taking subscriptions, then your need for security is that much greater - if someone is paying you for a service then you have to do your best to ensure that service is provided.

A centralised private cluster is far less susceptible to attack than a distributed one on a basically insecure network. Another method of promoting trust might be that the 'server-enabled' versions of the game are tracked - by unique ID per installation - so you can permanently disable them from cooperating in the game if they are compromised. This would be through your standard copy-protection key code, same way as Gamespy and many other cenral-server lobbies do it.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
To create a true p2p mmorpg the player locality principle can be used to run the simulation of a zone on all computers that have their players in that zone. This way each zone has as many redundant copies as many players are there. To keep players from cheating when they are alone (and noone else is checking their actions), each zone must be run on a number of backup computers, possibly running another zone actively. In the resulting system, the only way to cheat is to rig all computers in a zone and all backups. If the backups are distributed randomly, cheating is only possible if more than 50 percent of the computers are cheating. Network splits and joins are a possible event, much like what happens with bigger mulitserver irc networks. In irc, channel joins sometimes create user collisions. The same thing can happen in a game where two shards are joied with players standing at the same place in their own shard. Player commands must be sent out in lockstep mode to avoid race conditions, but this can create lag, because the reaction speed within a zone will be equal to the ping time between its two slowest and farthest computers.

Viktor

ps: P2P mmorpgs are a possibility, but they will have higher lag and become less secure than games based on centralized servers.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this