Server design problems

Networking and Multiplayer Programming

Started by Acar May 30, 2016 07:07 AM

4 comments, last by Basket 7 years, 10 months ago

1,159

Author

May 30, 2016 07:07 AM

I'll try to give a quick summary of how the server works currently then get to problems I have with it. It's a multithreaded asynchronous server, using IOCP to handle network operations. I do have bunch of managers like "Party manager, skill manager, item manager, trade manager, etc.". These managers have a one-way relationship. For example, trade manager knows about item manager but item manager doesn't work about trade manager. Until recently it's been working quite well. But lately I've been running into issues where 2 managers need to know about eachother. For instance:

- Combat manager needs to know about event manager so that it can add movement, skill cast events if the target is out of range.

- Event manager needs to know about combat manager so that it can call the function which handles skill cast when the target is within range.

One solution I could find was to move the function which handles skill cast into another class like "CombatUtility" and pass that to both event and combat managers. But it doesn't feel right and I couldn't think of a better solution. So my question is, how can I fix such problems, how can I avoid having such problems in the first place?

hplus0603

11,916

May 30, 2016 05:17 PM

- Event manager needs to know about combat manager so that it can call the function which handles skill cast when the target is within range.

This is where interfaces come into play. Event Manager doesn't need to know about "combat manager" specifically; it needs to know about "all the possible event receivers identified by name."

So, each event receiver tells the event manager what their name is. I don't know what language you're using, but all the languages support some kind of interfaces through some kind of polymorphism.

So, in C++, it might look like:

class IEventHandler {
public:
    virtual void OnEvent(PlayerId sender, EventId type, DataBuffer const &data) = 0;
};

void EventManager::RegisterHandler(std::string const &name, IEventHandler *handler) {
    handlers_[name] = handler;
}

bool EventManager::DispatchEventFromTo(PlayerId sender, std::string const &name, EventId type, DataBuffer const &data) {
    auto handler(handlers_[name]);
    if (handler == handlers_.end()) {
        return false;
    }
    handler->OnEvent(sender, type, data);
    return true;
}


class CombatManager : public IEventHandler {
public:
    /* other stuff goes here */

    void OnEvent(PlayerId sender, EventId type, DataBuffer const &data) override;
};

void CombatManager::SetEventManager(EventManager *mgr) {
    mgr->RegisterHandler("CombatManager", this);
}

void CombatManager::OnEvent(PlayerId sender, EventId type, DataBuffer const &data) {
    /* handling events go here, typically using some table that maps "type" to functions */
}

Obviously there are a lot of assumptions here, like the various types you actually use, and that managers are identified by strings. Another option is to set aside different event type value ranges for different managers, or use a registry negotiated at connection, or a number of different options with different strengths/weaknesses.

The main point here: Using abstract interfaces, dependency graphs can be made acyclic.

For what it's worth: The next step up in software organization is to define each of your services ("managers") in terms of interfaces as well, and use a registry of those interfaces. That will allow you to select a "Fake Network Manager" implementation when you want to run on a single machine, and a "Fake File System Manager" when you want to run unit tests fast.

enum Bool { True, False, FileNotFound };

Acar

1,159

Author

May 30, 2016 07:38 PM

Thanks for the great answer and the sample code. I am using C++ and I've had an almost identical approach with having handler functions for events. I've been using event type enums instead of event names though. Different part is that I had these handlers inside of event manager where I calculated the distance before calling the actual skill cast handler. Although interface approach seems much cleaner. Thanks.

Basket

199

June 03, 2016 04:42 AM

I'll try to give a quick summary of how the server works currently then get to problems I have with it. It's a multithreaded asynchronous server, using IOCP to handle network operations. I do have bunch of managers like "Party manager, skill manager, item manager, trade manager, etc.". These managers have a one-way relationship. For example, trade manager knows about item manager but item manager doesn't work about trade manager. Until recently it's been working quite well. But lately I've been running into issues where 2 managers need to know about eachother. For instance:

- Combat manager needs to know about event manager so that it can add movement, skill cast events if the target is out of range.

- Event manager needs to know about combat manager so that it can call the function which handles skill cast when the target is within range.

One solution I could find was to move the function which handles skill cast into another class like "CombatUtility" and pass that to both event and combat managers. But it doesn't feel right and I couldn't think of a better solution. So my question is, how can I fix such problems, how can I avoid having such problems in the first place?

Could you please describe your architecture in greater detail ? :)

If I understand correctly your server basically defines a internal publish-subscribe style of messaging system that allows " modules " of game logic to interact by exchanging messages in a loosely coupled way.

The advantage of such a architecture seems to be that gameplay code can be decomposed or subdivided into separate and self-contained units with private and internalized functionality and data. Interactions between gameplay code can be represented by the synchronous or asynchronous exchange of messages. These messages should be abstract and self-defining, in which case they are basically abstract and atomic units of information concerning events or requests or commands. The messages effectively describe the interface between the various units of gameplay code.

With such an architecture you can more easily add or remove or change gameplay code. Just add or remove or change the relevant " module " as desired, just be sure that the the flow of events and the information contained in the flow of events is not somehow meaningfully changed ( e.g. removing a " module " causes certain events not be emitted or certain information not be communicated that other " modules " are dependent on ).

I have read a little bit about these kinds of publish-subscribe server architectures. Mostly in the context of web services I think. As for MMO games, I know that " PlaneShift " uses a similar architecture but aside from them, I have no clue :(

So I sort of know the theory in a vague sort of way .... but I don't know how to architect these " modules " or the event flow between " modules " :(

Are there code bases I could study ? Or maybe could you describe the execution flow between some of your " modules " ?

( I am not looking at PlaneShift as its under the GPL <_< )

For example, the interface or API of and messages received by and sent by the " party manager " or the " trade manager " or both ( ^_^ ) and may be the general execution flow of the server ?

Does your event system send events synchronously or asynchronously ? That's another grey area for me :rolleyes: , I would imagine that its all synchronous messaging, basically amounting to chaining method calls.

Any insights you might be able to share would be really appreciated :D

Acar

1,159

Author

June 03, 2016 06:12 AM

@Basket

I'm afraid my server design isn't as well thought. In most cases this is the flow of the server:

IOCP->(packet)->protocol->(decrypted_packet)->handler_function[still in protocol]->(deserialized_data)->manager_function

Ex:

IOCP->(packet)->protocol->(decrypted_packet)->on_party_kick->(deserialized_data)->partyManager_->OnKick(Player *leader, const std::string &member_name);

Depending on packet type protocol might just do the complete handling of the message and not intercept with other managers.

Some of these managers also have 'Simulation' threads besides the handler functions. For node manager it checks if players changed nodes, for session manager it checks/updates item durations/cooldowns and other sessional data.

For events I have a C linked list. Besides from adding/removing events I don't have a synchronization. Unless the 'head' of linked list is being removed there's no 'waiting' while adding events. I do have internal synchronization for sensetive data like player stats, player lists and also internal synchronization for some of the managers.

Basket

199

June 04, 2016 04:19 AM

Thank-you AcarX for taking the time to describe your architecture :D

That is a real self-explanatory diagram :D

Hmm I actually was writing a long post, I was more confused than anything because I had some weird misunderstandings :D

But then I took the time to re-read your original post and the advice provided by hplus and then it really became clear :D

I think with the advice provided by hplus is pretty sound, I believe PlaneShift uses a very similar publish-subscribe architecture.

You can read more about it here : http://www.crystalspace3d.org/downloads/conference_2006/planeshift_conf.pdf

The interesting architectural stuff starts on page 24.

There is also a presentation here :

Server design problems

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Server design problems

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines