Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 14 Apr 2013
Offline Last Active May 30 2016 01:38 PM

#5234106 MMORPG networking Solutions

Posted by on 10 June 2015 - 12:05 PM

It's been a while since I've mentioned it, but it's come a long ways and I just released a new version.


Game Machine.


Game machine is an open source server platform that excels at large virtual worlds with lots of players in the same area.  It's focus is on solving a core set of hard problems not just networking.  It has built in persistence that scales well.  The best space optimization of any platform I'm aware of, and a well defined structure for writing game logic that runs concurrently.  Plus a bunch of built in functionality such as multiple characters, groups/chat, area of interest, etc..


The default client is for Unity.  Biggest market it's where I put my focus for all the client side implementation.  But it's fairly simple to integrate with the server as the core protocol is protocol buffers, and I moved stuff like reliable messaging to a higher level of abstraction.  So the networking layer is very dumb (as it should be IMO).


I wrote a completely functional open world mass pvp combat mmo in under 3 months using it, it was an eat your own dogfood project.  And all of the server side code for that is also open source and comes with.  So there are a ton of examples for how to solve most of the things you might tackle in an mmo.  

#5171222 Pathfinding in a 3D RTS or MOBA

Posted by on 02 August 2014 - 11:16 PM

raycasting can't give you paths for things it can't see, so it's really limited to being useful for local avoidance.   You are going to need a combination of pathfinding and steering to make a moba/rts game.


There are actually surprisingly few good open source pathfinding libraries, and TONS of crap written by people learning how to write pathfinding code.  My suggestion would be to just go with recastnavigation.   It's pathfinding is great.  The only part that's a bit dated is the crowd stuff.  It works but it's cpu intensive.  You can use it for a moba game, but for an RTS game with a lot of objects, it won't work as it just uses too much cpu.


My preference would actually be to just use an engine like Unity or UDK that already has what you need built in.  Even though their pathfinding is loosely based on recast, they provide crowd implementations that are far more performant and can handle much larger crowds.

#5159894 MMOs and modern scaling techniques

Posted by on 11 June 2014 - 06:09 PM


If a client wants to send 10 gold to someone and sends a request to do that, the client has no way of knowing if the request was processed correctly without an ack. But the ack can be lost, so the situation where you might have to resend the same request is present in all networked applications.


I think we're crossing wires a bit here. Reliable messaging is a trivial problem to solve (TCP, or a layer over UDP), and thus it is easy to know that either (a) the request was processed correctly, or will be at some time in the very near future, or (b) the other process has terminated, and thus all bets are off. It's not clear why you need application-level re-transmission. But even that's assuming a multiple-server approach - in a single game server approach, this issue never arises at all - there is a variable in memory that contains the current quantity of gold and you just increment that variable with a 100% success rate. Multiple objects? No problem - just modify each of them before your routine is done.


What you're saying, is that you willingly forgo those simple guarantees in order to pursue a different approach, one which scales to higher throughput better. That's fine, but these are new problems, unique to that way of working, not intrinsic to the 'business logic' at all. With 2 objects co-located in one process you get atomicity, consistency, and isolation for free, and delegate durability to your DB as a high-latency background task.



So this is an interesting topic actually.  The trend is to move reliability back up to the layer that defined the need in the first place, instead of relying on a subsystem to provide it.  


Just because the network layer guarantees the packets arrive doesn't mean they get delivered to the business logic correctly, or processed correctly.   If you think 'reliable' udp or tpc makes your system reliable, you are lying to yourself.





#5159657 MMOs and modern scaling techniques

Posted by on 10 June 2014 - 10:04 PM

Ok so I'll chime in here.


Distributed archtiectures are the right tool for the job.  What I've seen is that the game industry is just not where most of the innovation with server side architectures is happening, so they are a bit behind.  That has been my observation from working in the industry and just looking around and talking to people.


When it comes to performance, it's all about architecture.  And the tools available on the server side have been moving fast in recent years.  Traditionally large multiplayer games weren't really great with architecture anyways, and for a variety of reasons they just kind of stuck with what they had.  This is changing, but slowly.


I think the game industry also had other bad influences like trying to equate the lessons learned on client side performance to the server. 


The secret to the current distributed systems is that almost all of them are based on message passing, usually using the actor model.  No shared state, messages are immutable, and there is no reliability at the message or networking level.  Threading models are also entirely different.  For example the platform I work a lot with, Aka, in simple terms passes actors between threads instead of locking each one to a specific thread.  They can use amazingly small thread pools to achieve high levels of concurrency.


What you get out of all that is a system that scales with very deterministic performance, and you have a method to distribute almost any workload over a large number of servers.


Another thing to keep in mind is that absolute performance usually matters very little on the server side.  This is something that is just counter intuitive for many game developers.  For an average request to a server, the response time difference between a server written in a slow vs a fast language is sub 1ms.  Usually it's in microseconds.  And when you factor in network and disk io latencies, it's white noise.  That's why scaling with commodity hardware using productive languages is common place on the server side.  The reason why you don't see more productive languages used for highly concurrent stuff is not because they are not performant enough, it's because almost all of them still have a GIL (global interpreter lock) that limits them to basically running on a single cpu in a single process. My favorite model now for being productive is to use the JVM but use jvm languages such as jruby or closure when possible, and drop to java only when I really need to.


For some of the specific techniques used in distributed systems, consistent hashing is a common tool.  You can use that to spread workloads over a cluster and when a node goes down, messages just get hashed to another node and things move on.


Handling things like transactions is not difficult I do it fairly often.  I use an actor with a FSM, and it handles the negotiation between the parties.  You write the code so all messages are idempotent, and from there it's straight forward.


Handling persistence in general is fairly straight forward on a distributed system.  I use Akka a lot, and I basically have a large distributed memory store based on actors in a distributed hash ring, backed by nosql databases with an optional write behind cache in between.  Because every unique key you store is mapped to a single actor, everything is serialized.  For atomic updates I use an approach that's similar to a stored procedure.  Note that I didn't say this was necessarily easy.  There are very few off the shelf solutions for stuff like this.  You can find the tools to make it all, but you have to wire it up yourself.


Having worked on large games before, my recent experiences with distributed systems has been very positive.  A lot of it comes down to how concurrency is handled.  Having good abstractions for that in the actor model makes so many things simpler.  Not to say there are not any challenges left.  You hit walls with any system, I'm just hitting them much later now with almost everything.

#5108891 Version Control and programming with a team

Posted by on 12 November 2013 - 11:34 PM

Github has private accounts, it's used by thousands of companies developing commercial software.  I think it's under $10 a month which gives you several private repo's.


One thing to remember is that github != git.  Github has a workflow and a lot of tools to make your life easier, and there are a ton of third party services that hook into github.  As far as platforms as a service for source control, no one really touches github, and you have to factor that in to your larger decision on which source control software to use.


Beyond that I would just say stick with a distributed source control system like Mercurial or Git.  If you want to be able to manage different branches and merge code without pulling your hair out, don't touch Subversion.

#5106466 Browser games magic

Posted by on 02 November 2013 - 10:50 AM

The answer totally depends on the type of game you want to make.


For some types of games Unity is by far the best choice.  I put together a complete mmo tech demo in unity for a project I was working on inside of a month.


Javascript and html5 are limited, again it depends on the type of game you want to make.  Javascript is a horrible language, that's something you will have to reconcile with and just deal with. For a good laugh when you have 5 minutes see this: https://www.destroyallsoftware.com/talks/wat.


I've been watching html5 for years.  Flash became popular primarily because of how bad javascript was, and how slow the html5 standards were progressing.  Nothing has really changed.  All that has happened is that people have managed to squeeze more out of a stagnant platform.   Html5 will never become what people want it to be.  It simply can't when it's hamstrung by a bad language and a standards body that's paralyzed by politics.  Platforms like Unity or chrome's native code abilities are examples of where I think the future of games on the browser are headed.


Flash is kind of a dying breed.  It's good for specific games but I would be really nervous about using it on any new large projects.



#5102094 Just how complex are AAA games?

Posted by on 17 October 2013 - 04:26 AM

On the other side of this, I see a lot of discussion in the gaming community lately about taking up languages other then C++.   You should go read up on some of John Carmack's thoughts, or even take a look at the guys who wrote Go, and why google is dumping a lot of their C++ code.


IMO large complicated C++ programs are not something humans can reason about effectively.    Bullet is pretty clean, it's complexity is just par for the course for a  large C++ library that's been around a while.


In my experience AAA games get more complicated then they need to be, because of the timelines that have to be met and the shortcuts that get taken.  It's almost impossible to not have some really crazy stuff after 2-3 years of tight deadlines.  That leads to code that is complicated to reason about when it really shouldn't be.  Not because of bad developers, but because you just can't crank out good abstractions around C++ code at that pace.  The number of C++ codebases in the game industry that people look at and say 'that's really good code' is tiny, I could probably count it on half a hand.  But on the other side it got a lot of games to production and satisfied millions of users, so who's to say what the best approach is?