Sign in to follow this  
cebugdev

a fully scalable server

Recommended Posts

Hi guys, I would like to ask idea from you guys on how to implement a fully scalable game server.. Im using Linux and sockets for the server. And we will limit the connection for each server box(PC server) to atleast a 1000 connection, if connection exceeds 1k then it will be re-routed/forwarded to another server by the gateway/loadbalancer. -How can we implement this? -How to forward client socket connection to different server? I understand that there is some gateway/loadbalancer that is needed but i dont have idea on how to implement balancing of socket connection to server. Can you guys give suggestions? Thanks

Share this post


Link to post
Share on other sites
Quote:
Original post by cebugdev
Hi guys,

I would like to ask idea from you guys on how to implement a fully scalable game server..
Im using Linux and sockets for the server. And we will limit the connection for each server box(PC server) to atleast a 1000 connection, if connection exceeds 1k then it will be re-routed/forwarded to another server by the gateway/loadbalancer.

-How can we implement this?
-How to forward client socket connection to different server? I understand that there is some gateway/loadbalancer that is needed but i dont have idea on how to implement balancing of socket connection to server.

Can you guys give suggestions?

Thanks


You specifications are a bit vague, but Ill try to help.

First off, before you even think about it, throw out the "thread per-client" model (you never specified you where taking this approach, but if you are, drop it). Large amounts of threading does not scale. At first it may seem like your server is running wicked fast, but as soon as new users connect, a lot of context switching begins to take place, and your server will grind to a halt.

Try to never use more threads than CPU Logical Units. Use message passing and async techniques. It will scale a lot better when you start getting hammered with connections.

---

As far as forwarding a connection/packets, Ill share the way I am implementing this for a project I am working on.

[img]http://img535.imageshack.us/i/cluster.jpg/[/img]

First thing to note is a mistake in the diagram that I have not fixed yet, which is, the Connection Server(s) and the Service Server(s) communicate back and forth.

In the diagram, you can see a frame around "Login Server", "Connection Server", and "Ping Server". These are the front-end servers that the client directly communicates with. The client never communicates with any of the other servers.

The Login Server is the first thing the client interacts with. Here it requests a "session key" through authentication. Meaning, the user would type in their username/password and if authentication was successful, it would send them a list of available characters, an "auth key" and a list of "reamls" to connect to.

Once the user selects a character, he will be connected a Connection Server. The information he needs to connect to one of the Connection Servers will be included in the "realm" list. When a user connects to a Connection Server, they will pass their "auth key." The Connection Server will then lookup if the client was given authority to connect, if not, the connection should be dropped.

The Connection Server is used solely for managing connections ( connect, disconnect, network stats ) and to forward game messages to the correct Service in the cluster to be handled. Meaning, when a client sends the Connection Server a message, it determines what type of message it is and forwards it to the corresponding Service Server.

So, if a client sent me a "begin crafting" message, it would go to the Connection Server, the Connection Server would determine "hey, this is a 'begin crafting' message" and send it to the the Service Server which hosts the implementation for crafting in the game.

A Service Server simply hosts Services the game provides. Ive separated my Services up based on "features." (i.e. I have a "crafting" service, "chat" service, "combat" service, "spatial" service, ect...) But the way you break down your Services would be up to you.

---

That is the (very) basic rundown of how my cluster works. Its hard to give a one-shot solution, because one doesn't exist.

Hope this helped.

Share this post


Link to post
Share on other sites
Quote:
Original post by cebugdev
-How can we implement this?
Read up on epoll. For anything under Linux that has the words "1000 connections" in it, you need that.

Quote:
-How to forward client socket connection to different server?

One way to do that, and probably one of the simplest, is to do it on the DNS level. Give the client asking for a server.atyour.domain.com a different IP address each time, randomly chosen or round robin, or whatever.

Share this post


Link to post
Share on other sites
The reason to limit at 1,000 would be if you have expensive game rules / simulation, or really underpowered hardware. A linux server using something like libevent or boost::asio should be able to do 10,000 connections, if all it's doing is checking validity and forwarding data between connected users (assuming there's some reasonable upper limit on the number of users who can see each other, like 200 or so).

However, a "fully scalable game server" can mean so many different things to different people. You'll need a little more information to make an informed decision:
- What are the rules by which users can interact?
- Do they have to be in the same physical zone?
- Do you support friends lists and guilds?
- Is there commerce that needs multi-party transactional commits?
- Do you simulate physics on the servers?
- How expensive are game rules?
- Is there server-side AI / NPCs?
- What are the latency requirements? Is it OK if messages get delayed 10 ms through the system? 100 ms? 1,000 ms?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this