Quote:Original post by hplus0603Quote:By allowing the system to forget some recent updates when there is a crash -- but still maintaining consistency -- the system is able to provide much better latency.
To make this clear (to me): Is what you're saying that, if I for reporting reasons mirror the state of the world out to a separate database system, and there was a crash, then my reporting system would be out of sync with the actual world state?
No, thats very much an over-statement.
There is a *small* chance that this could happen, limited to the window of time between when the transaction coordinator calls your external DB service to commit its transaction and before it completes its own commit to the Data Store.
Quote:
And if I have to mirror all commits out to a reporting system anyway, why wouldn't I just use that system for my datastore/back-end? Scaling reads is easy, compared to scaling writes...
Because such an external data store would not be fast enough nor execution-aware enough to provide the data needed on the fly to handle events.
This is the same confusion Arbitus and I resolved above. The PDS Datastore is neither designed nor intended to replace a SQL database for statistical tracking.
What it replaces is the typical in memory model of game state. By doing so it makes that game state fault-tolerant, reliable, persistent and scalable across many cooperating computers. As you yourself pointed out, scaling writes is hard and the typical duty cycle on the in-memory model is 50% read/50% write.
That is why we needed our own technology and we had to cut anything that burned time that we didn't need to make that replacement.
The datastore's transactional nature, combined with the event execution model, means that game code that is written as if it was running on a single CPU can be farmed out across all the processors in all the boxes in the back end in a race-proof and deadlock-proof manner without any explicit synchronization on the part of the game programmer.
This is the most important thing about the system and was my original design goal when I started the project. It allows you to write highly parallelized and fault tolerant code without any knowledge of how it is being processed.
The rest of the advantages such as fail-over and durable persistence just "fell out" of the design.
Edit: If it helps any try this... the PDS as a technology does not compete with MYQL. Its closest competitor is probably Terracotta though each of those technologies do things the other doesn't.
Edit2: This might help too. Looking *just* at the Datastore and the functions it provides to the rest of the system, its most like a "tuple-space." The PDS execution model can be categorized as a "flow of objects" model, but with some very unigue twists. Unfortunately the existing tuple-space products we looked at did things we didn't need and didn't do thing we did, or we would have used one of them.
[Edited by - jeffpk on September 14, 2009 1:18:05 PM]