Jump to content
  • Advertisement


  • Content Count

  • Joined

  • Last visited

Community Reputation

120 Neutral

About VFe

  • Rank
  1. I have two questions: 1) Have you actually tried it? I've measured those document stores pretty extensively, and I concluded they're not up to that task. Probably because they're not designed for that task. You might as well say that Oracle and DB/2 can scale here, too, because they also support distributed transactions. 2) If you get "horizontal scalability" but at the cost of 1,000x less individual efficiency, then do you think that a business based on that will actually survive? The reason it works for web is that 99.9% of the time, cross-user interaction at low latency isn't important. For games, 99.9% of the time, it is important. Or, to put it another way: Do you think that Google Talk/Hangouts, or Microsoft Messenger, or Skype, use "web technologies" for the real-time chat parts? Do you think the brokering of an AIM message goes through Cassandra?       #1. Yes. Up to what task specifically? Speaking of Riak, since I have the most experience with it. Assuming you have adequate provisioning, it's fast enough to write/read within a single frame from server <--> ring, several; depending on how your cluster is set up, if you're using TCP, PB, or a custom Infiband driver. We're not talking concrete numbers here and that significantly impacts the nature of the discussion we're having, so please use some if you're going to attempt to refute every claim. This discussion requires context, state your perception of "loose physics", because it's possible given the correct constraints.   Oracle <-> Riak/Cassandra/Dynamo is a false equivalence, Distributed transactions in Oracle are about forwarding across data layers, Distribution across a Riak cluster is about providing fault tolerance and improving read access through denormalization. So no, I may very well not say that, because it is false.   #2. You're providing an extremely exaggerated number bordering on reductio ad absurdum. In the context of comparing it to Redis. It's an order of 2-3x average case, order of magnitude in worse cases, without application specific optimizations. Which again, depending on the latency tolerance of a particular application, may very well be acceptable in exchange for the ability to scale to hundreds of thousands to millions of users efficiently(at least as far as storage/retrieval go). If you're business teetering on the brink because managing 10 servers versus 3-4 is the breaking point(or 100 vs 40...or whatever scale you choose for that ratio), then your problems as a business are elsewhere, in marketing, sales, market fit...etc. Cause operation costs should not impact your survivability at those ratios.   As to the last point, I'm not even trying to argue that, but the question is "How much?" If your latency tolerance is above X, then these techniques can work. If it's not, than they won't, and I've never claimed otherwise. So you have to determine what that number is for your application before you can discuss techniques.   #3. No of course I don't. I fail to see the value in this assertion, you're making an absurd claim as to what I believe is feasible by comparing voice/video systems that have hard lock-step synchronized real-time constraints vs the soft real-time constraints we're discussing and that are common to a lot of games(including traditional MMOs).   I'm not making any claims that the things I've said are the best techniques in the world for every occasion, but you seem to be arguing that because they don't fit a very specific criteria for particular type of undefined game, that their value is near useless. Which is "throwing the baby out with the bathwater".   Frankly, I no longer think this is a constructive conversation amongst peers, and will avoid further participation due to professionalism.
  2. That's an issue with your specific example. Redis will have that issue, Riak , Cassanda, Dynamo won't. You trade off some latency, but gain high consistency and horizontal scaling.
  3. For the situation you're describing  there Kylotan, you may be better off with a distributed STM approach rather than thinking of it as "purely async". They're not mutally exclusive.   http://avout.io/ is an example of something I'm using in development right now.   It's very similar to MPI approaches for cluster computing.
  4. 3) The storage throughput is terrible. If you write anything to disk it's a colossal pain. Unfortunately much friendlier providers(DO,Linode) storage pools are tied to server size, with an absolute max.   Also, I won't argue data-center to data-center speed. You're absolutely correct. I wouldn't personally ignore last mile infrastructure though when discussing gaming, which is assumption I made in my response.
  5. I'm not sure if I'm misreading you. But I feel like what you said is very misleading. The real world performance of network infrastructure is not even slightly approaching 50% light. We typically max at 20% in best case scenarios. The majority of transit time is eaten up by protocol encoding/decoding in hardware, and improving the hardware or the protocol can dramatically increase transit latency. Ex. Going from tcp to infinband inside a cluster can reduce latency from 2milliseconds to nanoseconds. Not saying it's practical by any means, but we're bound by switches/protocols far more than light.
  6. I'm in a somewhat similar situation, I'm currently involved in design of scalable game systems. I come from a "big data" devops background, and have been facing many of the same issues. I have the same impression you got, that what us guys writing "large applications" do is completely alien to most game developers who only have game dev backgrounds.     My take-away so far:   The game itself being designed to be scalable is probably the biggest thing. GIGO. Some types of game-play are just inherently bad for scalability. So you're left with a choice of latency vs scalability. In many cases, there's ways to increase the complexity of events to break them apart to increase their scalability. In situations where this is asymptotically impractical or impossible, you need to design out the local machine state. That is to say, if events must be processed sequentially, design them so that the sequential elements can be processed on any server. Borrow from REST where you can. The trade-off here is you're often increasing your processing or network over-head as much as 3x...but honestly, it's often worth it. It's counter-intuitive, especially for game developers to design purposefully wasteful systems. Another problem for game devs in particular, OOP(design and principle, not necessarily the languages), is the enemy. It requires an extreme paradigm shift, the more state you can design away, the more scalability you get.   We're using an system very similar to what you're describing, asynchronous message passing across geographically diverse clusters(implemented in Clojure and Erlang, if anyones curious, with Riak as our back-end database), for large parts of our game system. And it works really well.   I'd say you can use these methods for just about any game system outside FPS, fighting games, maybe racing. Anything that is super latency sensitive is going to end up being a bad choice, but for everything else, I think it's a major win.         This imho is actually a really powerful method for games, similar to how we're doing things. Where every action is recorded to persistent storage(Riak), and then other players are essentially reading other players writes in a sense. With the memcache or application layer managing it.   Depending on the DB setup, this can be fast enough for real-time, but as with anything it's a tradeoff ala CAP theorem. For us, we decided that a little bit more latency in exchange for more scalability was a worthwhile tradeoff.
  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!