Btw... not to belabor the "experienced" point BUT...
http://www.linkedin.com/pub/jeffrey-kesselman/1/250/440
http://www.linkedin.com/pub/jim-waldo/1/53/342
In general today we majoratively see two kinds of users of the PDS. One is indeed small shops and indy's who are looking for something free. The OTHER equally strong demographic is academics who understand the real issues of massively scaling collaborative computing over distance, and game developers who have done one MMO and NOW understand them as well.
We are using PDS right now at Blue Fang Games.
(We make some products you might have heard of... Zoo Tycoon?)
'nuff said on that IMO.
Project Darkstar
Oh, by the way, in the interests of full disclosure..
I invented the Darkstar model and built the prototype, though the team at Sun Labs being deep experts in all their individual fields took that prototype and built an amazing piece of production software that is much better then I could have ever done alone.
It's genesis actually predates Sun's involvement. It was a project that I birthed on my own time while I was the Senior Game Integration Engineer at Total Entertainment Network (TEN.) For those who might not remember TEN, it was one of the first providers of high quality internet gameplay for packaged games. We were the online providers and wrote networking code for games such as Duke Nukem 3D, Total Anihiliation,Dark Sun Online and Nascar Racing Online (NROS).
While at TEN I realized we needed a tool we didn't have and i started building experiments on my own time. The third generation of those experiments, I showed to Sun Labs, and that was when Sun labs picked it up and has been running with it ever since.
All in all the PDS server has had about a decade of development put into it.
I invented the Darkstar model and built the prototype, though the team at Sun Labs being deep experts in all their individual fields took that prototype and built an amazing piece of production software that is much better then I could have ever done alone.
It's genesis actually predates Sun's involvement. It was a project that I birthed on my own time while I was the Senior Game Integration Engineer at Total Entertainment Network (TEN.) For those who might not remember TEN, it was one of the first providers of high quality internet gameplay for packaged games. We were the online providers and wrote networking code for games such as Duke Nukem 3D, Total Anihiliation,Dark Sun Online and Nascar Racing Online (NROS).
While at TEN I realized we needed a tool we didn't have and i started building experiments on my own time. The third generation of those experiments, I showed to Sun Labs, and that was when Sun labs picked it up and has been running with it ever since.
All in all the PDS server has had about a decade of development put into it.
I find the tone of this discussion thoroughly disappointing.
Despite supposed assumption, not everyone here is a hobbyst, not everyone is a beginner, and not everyone is clueless of architectural and engineering challenges faced when designing large, real-time systems.
Having been part of Java community for many years, it was ultimately the Trophy Generation that drove me away.
References are important. But not every project is plastered all over internet for everyone to make up their mind, yet this does not diminish the effort that has gone into them, nor challenges overcome and lessons learned.
Sadly, it is common these days to justify the merit through assertion of authority and attempts to undermine a persona, rather than presenting justified technical arguments which speak for themselves, and which others can independently evaluate. Some developers who actually use existing technologies do not or cannot understand the fundamentals, but some does not equate everyone.
This is about one single product. Even if this product were direct descendant of other projects, the best proof would be demonstration of how past issues were solved in improved versions.
And still, I have no problem with the project, nor people involved or effort spent. But the bully tactic demonstrated here is simply disappointing for people claiming to be experienced engineers. The attitude demonstrated here is worse than that of freshly graduated intern.
Why not let the project speak for itself?
Despite supposed assumption, not everyone here is a hobbyst, not everyone is a beginner, and not everyone is clueless of architectural and engineering challenges faced when designing large, real-time systems.
Having been part of Java community for many years, it was ultimately the Trophy Generation that drove me away.
References are important. But not every project is plastered all over internet for everyone to make up their mind, yet this does not diminish the effort that has gone into them, nor challenges overcome and lessons learned.
Sadly, it is common these days to justify the merit through assertion of authority and attempts to undermine a persona, rather than presenting justified technical arguments which speak for themselves, and which others can independently evaluate. Some developers who actually use existing technologies do not or cannot understand the fundamentals, but some does not equate everyone.
This is about one single product. Even if this product were direct descendant of other projects, the best proof would be demonstration of how past issues were solved in improved versions.
And still, I have no problem with the project, nor people involved or effort spent. But the bully tactic demonstrated here is simply disappointing for people claiming to be experienced engineers. The attitude demonstrated here is worse than that of freshly graduated intern.
Why not let the project speak for itself?
I answered the experience question.
I didn't raise it.
So if you have an issue with that, I'd suggest you take it up with the person who raised it to start with.
And the project's community and users *do* speak for themselves. Come and hear!
I didn't raise it.
So if you have an issue with that, I'd suggest you take it up with the person who raised it to start with.
And the project's community and users *do* speak for themselves. Come and hear!
Quote:Original post by CMelissinos
Sorry but, I really have to laugh at this. If you took some time to find out about the engineers behind this you would understand how ridiculously naive your comment is...seriously.
I don't think you really understand what the word "naive" means. You can have the greatest experts of all time come together and produce a "naive" solution. In fact, expertise and experience of the implementor are irrelevant when discussing the naivity of a solution. Here is a brief definition of the word for reference:
1. having or showing unaffected simplicity of nature or absence of artificiality; unsophisticated; ingenuous.
The Darkstar persistency model, unless you can provide evidence to the contrary, is absolutely the most naive solution I can think of for object state persistency. Storing serialized blobs with transactions after each update? That is the very essence of simplicity and extremely unsophisticated. Just because there may be some sort of voodoo magic under the cover implemented by the most genius man on earth does not change the naivity of the solution.
But, often the naive solution has drawbacks, and considering the large amount of available information regarding commercial MMO development that specifically deals with why such a solution was dropped in favor of alternatives, I would have to say that there is significant burden to overcome when making broad claims of the Darkstar object persistency model. Some issues have already been raised concerning the implementation that I believe are fair and valid, and are coming from experienced professionals in the field.
Quote:Original post by Antheus
I find the tone of this discussion thoroughly disappointing.
Despite supposed assumption, not everyone here is a hobbyst, not everyone is a beginner, and not everyone is clueless of architectural and engineering challenges faced when designing large, real-time systems.
Completely agree. However, some of the information being delivered here is mis-informed, but is being presented as fact. I don't think anyone here stated that anyone is clueless, just asking for their expertise to understand how they arrived at their conclusions.
Quote:Having been part of Java community for many years, it was ultimately the Trophy Generation that drove me away.
References are important. But not every project is plastered all over internet for everyone to make up their mind, yet this does not diminish the effort that has gone into them, nor challenges overcome and lessons learned.
Sadly, it is common these days to justify the merit through assertion of authority and attempts to undermine a persona, rather than presenting justified technical arguments which speak for themselves, and which others can independently evaluate. Some developers who actually use existing technologies do not or cannot understand the fundamentals, but some does not equate everyone.
I agree. However, hplus0603 used his own product as an example to the contrary. As Jeff has provided technical arguments for the platform, they were discarded with hand waving and pointing to another technology that the moderator has a vested interest in.
Quote:This is about one single product. Even if this product were direct descendant of other projects, the best proof would be demonstration of how past issues were solved in improved versions.
COMPLETELY agree and hope we can move forward from this and have a kick ass discussion about the issues that Darkstar is attempting to solve. To be perfectly clear, I have ZERO issues with criticisms, bring them on, just back it up with examples so we can address them. You won't find us hand waving and we are a very transparent group.
Quote:And still, I have no problem with the project, nor people involved or effort spent. But the bully tactic demonstrated here is simply disappointing for people claiming to be experienced engineers. The attitude demonstrated here is worse than that of freshly graduated intern.
Why not let the project speak for itself?
Again, what bully tactics? I mean, to be fair, others claimed that experienced users wouldn't use this, that Darkstar is focused on ease of use over it being robust, game developers have solved the things Darkstar is doing, etc. Again, I have no issues with anything that was said as long as examples are provided to back them up. I don't think that is unreasonable or bullying. Do you?
[Edited by - CMelissinos on September 11, 2009 12:20:07 PM]
Quote:Original post by arbitusQuote:Original post by CMelissinos
Sorry but, I really have to laugh at this. If you took some time to find out about the engineers behind this you would understand how ridiculously naive your comment is...seriously.
I don't think you really understand what the word "naive" means. You can have the greatest experts of all time come together and produce a "naive" solution. In fact, expertise and experience of the implementor are irrelevant when discussing the naivity of a solution. Here is a brief definition of the word for reference:
1. having or showing unaffected simplicity of nature or absence of artificiality; unsophisticated; ingenuous.
Of course I understand what naive means. But thanks for the reminder :)
Quote:The Darkstar persistency model, unless you can provide evidence to the contrary, is absolutely the most naive solution I can think of for object state persistency. Storing serialized blobs with transactions after each update? That is the very essence of simplicity and extremely unsophisticated. Just because there may be some sort of voodoo magic under the cover implemented by the most genius man on earth does not change the naivity of the solution.
But, often the naive solution has drawbacks, and considering the large amount of available information regarding commercial MMO development that specifically deals with why such a solution was dropped in favor of alternatives, I would have to say that there is significant burden to overcome when making broad claims of the Darkstar object persistency model.
Any pointers that you have to these claims would be appreciated.
Quote:Some issues have already been raised concerning the implementation that I believe are fair and valid, and are coming from experienced professionals in the field.
Again, please provide a pointer to these issues and who the experienced professionals are in this field. I am genuinely curious as to the references you are citing.
Happy to be proven wrong, no issues with that. Just would like to know the validity of the source(s).
Hi, I'm a noob. I'm not a systems programmer, or an expert game developer, but I am investigating Darkstar. I have been active on the Darkstar forums for a couple of months.
I am actively researching MMO development, and all the processes involved for a proposed project at a Uni course I am about to embark on. I've been attracted to Darkstar because of its ease of use, and the very thorough and precise technical information that is available on the Darkstar forums.
What I am interested in, and hope to recieve here, is a precise technical overview of why Darkstar's Datastore isn't a good idea. To be honest, I've done a bit of programming myself, querying ManagedObjects by using the ScalableList or ScalableHashMap class and Managed References. It seems to me that although the data is stored in serialized blobs, I can build code to access that data, and get its last known state. Now ok, I have no history of previous states, but I have the present state. I get the impression that that is the whole point of Darkstar's datastore, to preserve state in the event of a crash somewhere in the cluster.
Now I've only been Java-ing for a few months, and it strikes me that I could do quite alot more with the datastore than just hold game state, but I am also aware that a MySQL database (or any other server based database I suppose) can be plugged in and synchronized with the Darkstar datastore via the services layer. So if I can use all the functionality of a MySQL database alongside Darkstars datastore, doesn't this solve the problem? That is IF the Darkstar datastore is actually a bad idea to begin with.
I'm hoping someone from the GameDev side of things can explain to an admittedly naive noob WHY this is Project Darkstar's datastore is a naive idea, and maybe suggest other implementations in a bit of detail.
And I ain't got no beef here, just trying to learn something new.
Cheers, Will.
I am actively researching MMO development, and all the processes involved for a proposed project at a Uni course I am about to embark on. I've been attracted to Darkstar because of its ease of use, and the very thorough and precise technical information that is available on the Darkstar forums.
What I am interested in, and hope to recieve here, is a precise technical overview of why Darkstar's Datastore isn't a good idea. To be honest, I've done a bit of programming myself, querying ManagedObjects by using the ScalableList or ScalableHashMap class and Managed References. It seems to me that although the data is stored in serialized blobs, I can build code to access that data, and get its last known state. Now ok, I have no history of previous states, but I have the present state. I get the impression that that is the whole point of Darkstar's datastore, to preserve state in the event of a crash somewhere in the cluster.
Now I've only been Java-ing for a few months, and it strikes me that I could do quite alot more with the datastore than just hold game state, but I am also aware that a MySQL database (or any other server based database I suppose) can be plugged in and synchronized with the Darkstar datastore via the services layer. So if I can use all the functionality of a MySQL database alongside Darkstars datastore, doesn't this solve the problem? That is IF the Darkstar datastore is actually a bad idea to begin with.
I'm hoping someone from the GameDev side of things can explain to an admittedly naive noob WHY this is Project Darkstar's datastore is a naive idea, and maybe suggest other implementations in a bit of detail.
And I ain't got no beef here, just trying to learn something new.
Cheers, Will.
Quote:I imagine his objectivity may be a bit biased.
There is no such thing as total objectivity, for sure!
The reason I'm talking about Darkstar is that I (in my role as CTO at Forterra -- which I didn't have when I become moderator on this forum) actually evaluated it in some depth, and found it wanting for our needs. Instead, we are using a traditional MySQL back-end for our data store. For full disclosure, we actually initially started out on Oracle, but found it also didn't suit our needs in the end, as it was much more expensive and complicated (hardware-wise) to scale.
Architecturally, our technology would largely sit on top of a system like Darkstar, rather than compete with it. Which means that we may compete with other users of Darkstar, for sure, but that doesn't mean that we wouldn't have used Darkstar if we had found it to be the best fit for our needs.
Part of the evaluation was talking to other people who tried to use Darkstar, including some gaming heavyweights, and in the end they all found it wanting. Unfortunately, most of that evaluation was "if you tell anyone else I won't talk to you ever again" type conversations over beer, so I keep those people's names under cover -- instead, I posit my specific, technical analysis. That, at least, can be more or less objectively analyzed in a specific context.
Regarding the Darkstar model of inhaling objects, running transactions on them, and then exhaling again, I don't think that's a new invention as such. For example, OODBMS vendors have been hawking that model in various guises for the last 20 years. However, I think it has been a source of interesting research, and I'd love to see more peer reviewed research publishings if there have been any really new algorithmic discoveries as part of the Darkstar implementation.
Btw, someone pointed out this quote from the Sun roadmap:
Quote:While it is possible that such an update could, on failure, leave the two durable stores in an inconsistent state, the window in which such a failure could occur was decided to be sufficiently small that repair by human intervention was an acceptable alternative to implementing a generalized transaction manager.
I'd like some more insight into this, if possible.
I'd also love to see a real technical comparison between Versant and Darkstar, btw -- do you Sun guys (now Oracle, I guess?) feel up to it? I think that would be valuable to everyone in this discussion.
I think this blog post by Jim Waldo is pertinent to the discussion, at least as a matter of context.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement