I've reached a turning point in my server application(s) building phase where it's becoming a more than reasonable idea to centralize some of the non-persistent data in my system(e.g. player positions, multiple systems need some access to recent positional data).  I presently have 6 individual server applications that interact with the game client to some extent and interact with each other to some extent.  The complexities of managing the data flow(s) on a peer to peer basis are quickly becoming a hindrance to forward momentum.  Additionally my present design doesn't scale, well at all.

I'm presently using key-value data types almost exclusively within the server code itself, to store the data that I need to centralize.

And I need NO persistence, there's no need for a database or file system behind the in-memory KV database.  All persistent data is handled by the server code directly.

For scalability it should be able to replicate/synchronize the data across the internet if need be.

So that leads me to a couple of choices and my question.

Which one?  Nearly ALL the feature comparisons I can find are published by Alachisoft, and thusly are biased against Redis, and a few years old from the look of them.  As well as not being particularly detracting from Redis as a candidate for my use case.

I'm looking at 3 options:

1. Ncache http://www.alachisoft.com/ncache/
2. Redis https://redis.io/
3. HomeBrew...   Some of the code I've already got + a good replication algorithm and I'm set.. hahaha  Not listed as #1 for a reason.  I'm just not sure I want to add an 8th piece of code to my present development load.

Does anybody out there have experience with either or preferably both of these cache/db servers? 

Presently I'm leaning towards NCache simply because it matches my present server development and my most likely initial server release environment of Windows/.Net/etc...  But I really like everything I've read on Redis as well, and I've engineered Linux servers for as many years as Windows....  And my eventual scale-up platform will likely be Linux..  So, there's most of the variables I'm considering.

I'm really mostly interested in anybody's practical experience with these two or any others in their immediate ball-park, I think Apache Ignite is feature similar with the above two...

Also interested in any non-biased 3rd party comparisons that I'm presently unable to find, if you have any quick links handy.

Thanks for reading!

Well, I guess the only way to be sure is to test them out anyhow.

I shall report on my findings, I just finished installing a Centos7 vm and I'm compiling Redis 5 as I type.  I'm also going to install the free/open version of NCache and see how they compare for code complexity/use.

The testing will have to wait 'til morning time though(almost 1am here).

Well, shoot, compilation and installation was about as smooth as just about anything I've ever had to compile myself on linux.. ever..   Good signs for Redis already.

I have continued this to my project blog:

https://www.gamedev.net/blogs/entry/2265641-in-memory-key-value-databases-redis-initial-evaluation/

If you wish you can follow along from there, I'll be evaluating NCache next.

Redis is great, as long as you don't need to scale past a single machine per game instance. If you don't need persistence, you could just use pub/sub and leave it at that.

Another option for pub/sub is ZeroMQ, btw. It's decidedly non-persistent, and you may need to build your own central broker that actually lets everyone else rendez-vous to pub-sub, if you don't want everybody to connect to an existing server.

But, more importantly: Why do you have multiple different servers that need to know player location? Why not build a single server process? The constant factor overhead in trying to build a "scalable" solution that distributes frequently-changing CPU-decided data (like physics simulations) across multple machines is gigantic. Game servers are not web applications. How about putting all code that needs player location onto a single machine, and just read the locations when you need it? Once you run out of vertical scalability on that machine (at 1000 or 10000 or 100000 players) then shard your world.

47 minutes ago, hplus0603 said:

Redis is great, as long as you don't need to scale past a single machine per game instance.

What makes you say that exactly?

47 minutes ago, hplus0603 said:

Another option for pub/sub is ZeroMQ, btw. It's decidedly non-persistent, and you may need to build your own central broker that actually lets everyone else rendez-vous to pub-sub, if you don't want everybody to connect to an existing server.

I will look into it, thanks!

47 minutes ago, hplus0603 said:

But, more importantly: Why do you have multiple different servers that need to know player location? Why not build a single server process?

I'm building a massive persistent online world in a "single shard" environment.

47 minutes ago, hplus0603 said:

The constant factor overhead in trying to build a "scalable" solution that distributes frequently-changing CPU-decided data (like physics simulations) across multple machines is gigantic.

Indeed it is gigantic.  But not quite as..  Only the position relay server needs subsecond resolution.  It or one of its clones will speak to the client(s) directly and will be populating this data into the database for other servers to reference.

47 minutes ago, hplus0603 said:

How about putting all code that needs player location onto a single machine, and just read the locations when you need it? Once you run out of vertical scalability on that machine (at 1000 or 10000 or 100000 players) then shard your world.

There will be no sharding in that sense.   I'm using a single data warehouse to hold all of the non-persistent data so that I can run clusters of functionally specialized servers.  The primary goal of which is to allow a shared reality to propagate across an internet sized network.

Check out my project blog if you want, I would love more feedback/suggestions/critiques.

My experience from massive simulations is that any attempt to "overlay" simulations (not clustering them geographically) will fail to scale, because the communications overhead (networking) is orders of magnitude higher latency and lower bandwidth than NUMA RAM. Giant persistent worlds are totally possible, and have been built for 20 years, but they all end up sharding by geography and replicating entities across shard borders in one way or another.

If you don't need any kind of physical simulation or physically-based rules (enemy targeting, spells, meelee combat, etc) then you can probably go with a fully sharded "web style" network topology, but then, what would your game be? The world can only see so many Hearthstones or Clash of Clans.

Anyway, I say Redis is great, because I've used it for a number of different use cases, and I've seen several other companies use it, and it's great as long as you stay within a single machine. Once you need a cluster of machines, many of the things that are good about Redis, such as transactional cross-key updates, stop working. If all you use is pub/sub, and you shard the pub/sub key space across separate Redis instances, that can scale higher, but it has the draw-back that you get NxM network connections -- each interested subscriber process needs to connect to each Redis instance, and because both Redis and subscribers scale by number of players, it ends up growing as O(N-squared) albeit with a very small "k" value.

Redis is less great at persistence, because once the data store gets big, you need machines with tons of RAM, and forking the Redis process to persist it will lock up the kernel for seconds at a time if you have enough RAM. You can somewhat alleviate this by sharding across tons of small Redis instances (say, 32 GB each or less) but then you instead have a bigger "M" problem for connections. Use a real database for persistent data.

Awesome, killer info!! 

Thanks!  Yeah, I guess there are a lot of different ways to use the word shard when talking about this stuff.. haha  I assumed it was mostly a term to describe the over-all effect of the experience.  In reality I guess my functional segmentation is actually a type of sharding all on its own.  So, my game universe will be a single shard experience, but the servers are sharded every way you could possibly imagine.

In order to avoid the static geographic clustering that you mention, I'm attempting to devise a system that is distributed and dynamic.  Instead of moving players from shard to shard as they move through the world, the "shards" will move with the players,  merging and separating from the shards of other players on the fly(by proximity).  It's all still highly theoretical and mostly only makes sense to me when my eyes are closed.  haha

My primary purpose for THIS system is to centralize transitory data so that other systems, shards if you will, can access it within a reasonable time frame(before it falls out of recent history, the short-term memory for my universe).  Without this, I'm going to have multiple clusters of servers spitting data at each other left and right, regardless if the receiver really needs it or not...   Not happy making.

Thanks again, I really appreciate the info!

@hplus0603 Have you looked at Riak at all?  It kind of "looks" like a more scalable version of Redis to me..

http://docs.basho.com/riak/kv/2.2.3/

I would be very interested to hear your take on it if you have one.  It has a memory-only back end option and is built to be clustered/redundant in even a minimal installation(3-4 servers).  I'm going to give it a spin next I think and see how it goes.

Thanks again!

43 minutes ago, frob said:

You talk of multiple servers, and you also talk as though you are a single individual.

Oh, if I had some better multiple personality jokes... zipzoombang!!!

I sense just a little bit of callus and skepticism in some of the replies I get to my questions these days.. haha.  I'm fine with that.

At the moment, yes I am a single individual, and yes I am writing multiple servers, 6 at the moment.  I have a partner joining in the project soon, he's also very capable, but not really a programmer.

43 minutes ago, frob said:

Pick a solution that fits your actual problem.  People pick something that is high performance for a different industry but terrible for games, or that scales to support thousands of machines when they won't ever need it.

A solution that has a capacity to scale, that exceeds the need, is what I'm trying to pick.   I'm simply trying to find something that will(eventually) scale, this kv database is just a tool to keep things in sync.  In fact it's just a really big centralized Dictionary(with extra functionality), not unlike the Dictionaries and other KV types I'm already using in my server code. 

43 minutes ago, frob said:

Consider your game at its most successful, but while still small enough to not hire anybody. How many transactions per second will it need to support? 

Realistically, for many games the peak rate is a single digit number of tiny transactions per second.  Even if the game were to become successful and make enough money to start hiring additional programmers, the peak rate for thousands of users can reach tens of transactions per second.

Indeed, I'm not building a global network of game clusters right now, but I am attempting to build my system in a way that can scale up to..etc.  As well as being reusable for future gaming projects or expansions on this one.  And I would like to start with a lot of extra capacity. 

43 minutes ago, frob said:

Typically games don't realistically need clusters of servers until they're also large enough to have large teams of developers.  When they grow to that point it is easy enough to hire somebody to write that code.  If your game is small enough you can't hire people to write and maintain databases, then it is also small enough that you don't really need that technology.

The game environment I've created is already about as massive as I could imagine, so in actuality I'm just trying to match my server architecture to the capacity of my game environment.  It's hard to imagine trying to populate a world full of things without the capacity to keep track of it all.

And, my entire background is in enterprise systems engineering, I can build and maintain servers and server clusters, I can build and maintain databases, this is stuff I do without it feeling like work.   I've also been programming in one language or another since I was a child and I'm almost 40 now.  So, I'm leveraging my knowledge and experience and building my dream project from the ground up.  Since I can write that code, there's no need to hire somebody else to do it.

43 minutes ago, frob said:

And if you're the person hired to add that to a game that is large enough to require such a cluster, you likely don't need that kind of tech until you are reaching a thousand transactions per second or more.  If you're in that boat, you'll have time to do a full evaluation of many technologies and probably wouldn't be asking here.

I am doing a full evaluation, and no I'm not being paid for it, the first part of that process(for me anyhow) is asking around...  

I'm not here to waste anybody's time, I'm providing rudimentary benchmark results for my evaluations(as I linked above) over on my project blog, in case anybody else gets crazy wild and wants to play with big tech too. 

If anybody else has some experience with THIS(in-memory scalable KV databases) technology I would love to hear about it.  But for the most part I think I've decided on Redis as my "currently workable" solution.  This is a pretty simple technology when it comes right down to it, and most of the players are so cross-pollinated that it's hard to tell their open source versions apart.  The enterprise versions are where they really differ, but I'm not really interested in those product offerings right now anyhow.

Thanks Again!