Hi all, I have been working on a simple framework for me to use in making my prototype client/server mmo. I would love some feedback on any flaws my approach may have. My framework basically has 3 classes, a kernel, a manager interface class which the kernel interacts with, and the basemanager class which all manager's inherit from. A manager is a class that has a specific task for instance, player manager deals with player objects and player_id mappings, network manager deals with receiving and sending network packets, etc... Each manager can run on it's own thread without any locks because I use message passing to communicate between each manager. At the beginning of each manager's update method is when the manager executes any message they have received. I also have each manager have a private and public state so other managers can use the manager's public state to get any data they need. For instance, if a physics manager needed a player_id to player object mapping, the physics manager would use the player manager's public state object. No locks are needed when getting a manager's public state and thats because each manager only update's their public state after all the manager's are done their update methods. Now my framework makes all the manager's wait until each manager is done their update method and then they all wait for until each manager is done their update state method. This would have a bottleneck of the slowest manager, but I think if I don't design a manager with too much work, this framework could be very scalable. And if a manager is taking too long, I imagine I can optimize that manager by using normal optimization techniques and even worker threads or even be able to split the entire manager into two separate ones. Right now, I make each manager give basemanager a ThreadingPriority enum (high, moderate, low). I then poll the amount of cores a system has and the manager with the highest ThreadingPriority gets priority being on their own core. So for example, if I have 2 cores on a machine, and 3 managers, the manager with the highest priority gets it's own thread, and the other 2 managers both run on the main thread. I probably didn't explain everything well enough so please let me know if you have any questions. I will paste my 3 classes here incase someone reading this finds that easier to understand what my framework does. Kernel: http://pastebin.com/m17acffcb IManager: http://pastebin.com/m28f1a7b0 BaseManager: http://pastebin.com/m24d29d30 PlayerManager(for an example manager implementation): http://pastebin.com/m542d807 Now this is a prototype so I haven't optimized anything, I just want to see if my concept is solid. Thanks all! [Edited by - link3978 on February 11, 2010 10:25:55 AM]

Quote:Original post by hplus0603
If your target is 100 players, then almost anything can work.

If your target is > 1,000 players, then you need to read up on data federation, sharding and work pipelining.

Thanks hplus. My target is > 1000 (prob 3k clients would be max) so do you have any good resources you can point me too about those things you mentioned? I'll google them and see what I can find but anything you can point me to would be great.

Thanks man = )

Also, are you saying to look at those things because my design is flawed? Or is my design good but I need to incorporate those 3 topics as well?

Quote:Original post by hplus0603
Your design is not a "design" as much as an "implementation strategy."
I think you need to approach the design of an operating MMO infrastructure at a much higher level first.
How you then choose to implement the design becomes of secondary concern.
For example: What kinds of data is persistent? How does it persist? What kind of consistency guarantees are there? How do you enforce those consistency guarantees? Etc.

I think I see your point. I haven't addressed how I would retrieve and store data in a database yet. What I got will work for in memory data, but how would persistence data be dealt with. Yes ... I see your point ... if thats what your getting at.

Quote:Original post by Codeka
Also, I assume you won't be expect 3k people at launch. It'll take some time for the user-base to build up. I would suggest you simply keep in mind that you'll be expanding, but don't stress the details too much. Instead, concentrate on your actual game first. Without a game, you're not going to have anything to entice those 3,000 people to come alone anyway!

Once you've got 100 or so regular players, you can start to see the kinds of stress they put on your system and start extrapolating from that to what you'll need for more.

I'm not really worried about my game yet. I find it more fun thinking up implementation strategies that are scalable but most importantly easy to develop in. I'll probably never actually complete my prototype game, I hopefully will someday have something playable though.

But I am mainly asking for feedback because I know some peeps that are restructuring their code-base and if my architecture I described looked good, i'd try to get them to adopt it. And they will definitely have 3k+ at launch = )

[Edited by - link3978 on February 11, 2010 7:28:52 PM]

This design, while probably easiest to reason about and most robust as far as total correctness goes, does have one fatal flaw.

Assume that 800 actions take 100ms to process. During these 100ms, 1000 messages queue up, which take 120ms to process. During 120ms, 1200 messages queue up and take 140ms to process.....

And since most of these actions require interactions with others, and cause side-effects, the increase in processing time will be between O(n) and O(n^2). This directly affects users, since their perceived response time will vary depending on how many total actions there are in the system.

So unless you put hard limits on how many actions can occur and in what way, or which may be discarded or delayed, there's a potential time bomb.

From user's perspective, who doesn't care even a bit beyond their line of sight, and the button they just pressed, the rest of actions do not exist. So whenever there is a fight going on in nearby pub, they'll be annoyed their Charge! is taking 5 seconds to activate.

Alternative is to split same type of simulation into separate units, but then you lose total consistency. This is likely the main reason why WoW is very lax on collision detection and positions and tolerates rogues abusing bandwidth throttling to fake their location. I have no clue if that still works, but I think that idea was that attacks were activated by client's view of world, so when they stunned someone, everyone saw them 50m away due to lag.

Of course, as far as consistency and such issues go, which came first: WoW becoming popular despite being lax on such rules, or people tolerating lax rules due to WoW being popular.

And while I haven't seen it myself, I heard that Aion doesn't show any lag problems. But it's hard to take such information from third-party, since most do not differentiate between graphics, network and simulation lag. It might however be that just limiting to 5 actions per player per second and 3000 players per shard is something that today's hardware can handle without any particular optimizations. There's more than one way to skin a cat.

Quote:Original post by Antheus
This design, while probably easiest to reason about and most robust as far as total correctness goes, does have one fatal flaw.

Assume that 800 actions take 100ms to process. During these 100ms, 1000 messages queue up, which take 120ms to process. During 120ms, 1200 messages queue up and take 140ms to process.....

And since most of these actions require interactions with others, and cause side-effects, the increase in processing time will be between O(n) and O(n^2). This directly affects users, since their perceived response time will vary depending on how many total actions there are in the system.

So unless you put hard limits on how many actions can occur and in what way, or which may be discarded or delayed, there's a potential time bomb.

From user's perspective, who doesn't care even a bit beyond their line of sight, and the button they just pressed, the rest of actions do not exist. So whenever there is a fight going on in nearby pub, they'll be annoyed their Charge! is taking 5 seconds to activate.

Alternative is to split same type of simulation into separate units, but then you lose total consistency. This is likely the main reason why WoW is very lax on collision detection and positions and tolerates rogues abusing bandwidth throttling to fake their location. I have no clue if that still works, but I think that idea was that attacks were activated by client's view of world, so when they stunned someone, everyone saw them 50m away due to lag.

Of course, as far as consistency and such issues go, which came first: WoW becoming popular despite being lax on such rules, or people tolerating lax rules due to WoW being popular.

And while I haven't seen it myself, I heard that Aion doesn't show any lag problems. But it's hard to take such information from third-party, since most do not differentiate between graphics, network and simulation lag. It might however be that just limiting to 5 actions per player per second and 3000 players per shard is something that today's hardware can handle without any particular optimizations. There's more than one way to skin a cat.

Thanks for the reply.

I'm not quite sure I follow you but if the server was getting bogged down, I imagine it would be from the heavy hitters like, collision detection and physics. Well if I had physics be it's own manager, then when the server is getting bogged down, I just don't call the physics manager's update method as frequently. I still however let it process any messages it receives from other managers but by calling it's update less frequently, it wouldn't run the costly simulation as much.

Does that address the problem you mentioned?

Thinking about all the mmos I've played, I think the main thing that everyone does most is walk/run around. So if this is the worst case (and it might not, i'm just thinking outloud now), then the collision detection/physics would be the most taxing component of a mmo server, right? So if the server is being bogged down, wouldn't it most likely be because of the CD/Physics? The only thing I see being anywhere close is database interactions. Thoughts?

[Edited by - link3978 on February 11, 2010 9:52:49 PM]