Quote:Original post by Antheus
This was a reply to other thread which got locked or deleted in between.

Quote:Original post by jeffpk
The flaw is that it misunderstands the Project Darkstar model. Minor state changes are simply communicated between clients over the channel system. Transactions are only invoked for game-state significant changes.

What if all changes are significant? For example, fully deformable terrain, full physics interaction?

The fact that MMORPG model can scale to 3000 users on single box is not new. Single core can support thousand, or more, many emulators prove that, the Ultima emulator written in C# supposedly holds that many. And as long as the world can be split into disjoint pieces, achieving thousands is not really a problem.

One piece of information that Jeff left out was that the 3000+ users on that single core box PEAKED at just 30-40% utilization and demonstrated near linear scaling across 4 cores. Much more headroom left in a single 4 core system.

And your example of an Ultima emulator written in C# does none of the things you asked in your first sentence such as fully deformable terrain or full physics interaction. So, what's your question?

Quote:

Quote:The proof of the pudding is in the tasting, as they say. You can see a demonstration of this model in action by downloading the "Project Snowman" demo action game from the PDS community site. It has been tested to scale to 3000 simulated users and 2000 AI opponents on a four-core AMD server.

I'm more impressed at the client being able to render 5000 actors at the same time. I know that even the best video cards have trouble rendering this many units. Or are those players not within range or visibility of each other?

Show me any client today capable of rendering 5000 actors at the same time. we are talking full resolution, fully aware, collision based actors. No engine today can do this on any complex models. And, none of this has to do with server side interaction.

Quote:Users metric doesn't say anything. Chat servers have 10,000 or more per thread.

Great. Darkstar was used in an graphical chat system for JavaOne and was tested way above 10,000 "players" on a single node. We tested internally to 20,000, no issues. And, the server code is available to download, for free of course.

Quote: What matters is the complexity of interactions and overall complexity of simulation. That post talks specifically about physics heavy simulation, which are not trivial to distribute.

The description makes the game sound like Scorch/Worms style gameplay. Those are trivial to scale, since each client only emits an action once every few seconds.

Of course it's not trivial to distribute, which is why no one does it in any significant resolution today. Terrain deformation, for example, is handled locally and state refresh is order of magnitudes behind the client state. Games like WURM online can attest to this.

Quote:Is there a server running anywhere, where one could join and test this live?

There are several public Wonderland links that can be found at Project Darkstar.com. One of the features of the new Wonderland system is the ability to add geometry to the world instantly, with refresh across all clients. As well, we are showing multi-node and performance enhancements at AGDC next week.

Quote:Edit:

">I found this. Seems to be an action per second for attack + movement and no collision detection. Picking up the flag and turning also seems to take a second or so. It looks as if each actor performs one action each second.

I'm not bashing the project, I'm just trying to figure out the exact simulation model.

Instead of looking at a video on YouTube, how about downloading and trying for yourself? :)

Quote:Original post by hplus0603
I have heard of people trying it but giving up on it. The main project actually using it is Wonderland, which is currently undergoing a major rewrite.
I don't like it because it doesn't support ad hoc queries, and you can't use it for real.time properties (only transactional state).

So, hang a SQL database off of it and query the item pool there. I have to question whether or not you have actually tried the technology recently.

Quote:Original post by hplus0603
Darkstar can serve as a back-end database, but not as the host for simulation. Suppose you have 10 properties on an entity (position, orientation, velocity, etc) that change 30 times a second -- that's 300 transactions per second just for that one entity! Last I heard, Darkstar has not scaled beyond 10,000 property updates per second; it's not designed for that kind of work.

"Last I heard" means all of your statements are tainted. Show me a game that is not an FPS that is utilizing 300 transactions per second, per entity. It's not the way MMOs work and it was never stated that Darkstar was designed for FPS's. Its that state load that is precisely why you do most of the transactions peer to peer and dead reckoning on the client.

Quote:Thus, you'll have to build your own property update and distribution system for anything real-time, negating a lot of the gain of using a pre-canned object/property/distribution system. Darkstar then becomes little more than an application server/database front-end.

I have to ask, have you actually used the technology yourself? I ask because almost everything you state above is inaccurate.

Quote:But, if that's what you use it for, then you typically want to do database-y things with it. Like, running reports and queries -- "How many users traded in a transaction worth more than 2,000 gold in the last week" and the like. Except the Darkstar data model is just an opaque blob of serialized data.

Darkstar is designed to maintain high performance transaction/object state. It is not a DB. Again, hang a SQL server off the back end for long term storage and retrieval.

[quote}You have to *write code* for each kind of query you want to run, and that code has to be present *during the entire time interval of interest* -- no going back and running new queries on old data! And each query you write code for additionally will add more load to the transaction throughput.

Quote:The classic architecture of a sim/game server that talks to a SQL database on the back end for persistence makes a lot more sense for almost all game scenarios I've run across.

You mean the same sharded, let's predict what the players will do, constantly falling over architecture that we have been using since UO? I know people cite Eve Online, which tops out at about 60,000 simultaneous players and required military grade RAM disks to achieve simple communication to all players, but that is like strapping a rocket to a Chevette and calling it a Porsche. The only game that can use that infrastructure is the single game it was designed for, EVE Online. Every single MMO of scale that used the old sharded method, including WoW, has the same problems with transaction integrity that Darkstar solves for.

Don't believe me? Start reading here: http://www.amazon.com/Exploiting-Online-Games-ebook/dp/B00142KQE8/ref=pd_sim_kinc_3

Quote: Locking and serialization is handled elegantly in Darkstar, but solving those problems is not that hard, and certainly not worth all the draw-backs IMO.

Here is where I have to question if you really know what you are talking about, sorry. If it were that easy, dupe bugs and client side exploitation of the server would have been eliminated a LONG time ago.

May I politely ask what your background is in server side development? As well, have you ever used the technology or just read about it? I am not trying to be snarky, just really would like to know.

EDIT: Ah! Just found it. You work as the CTO for Forterra Systems and have your own, home grown back end system. So, the question remains, have you actually tried Darkstar or just basing your assumptions on it vs. the system you have built for Forterra?

Quote:Original post by hplus0603
I have heard of people trying it but giving up on it. The main project actually using it is Wonderland, which is currently undergoing a major rewrite.
I don't like it because it doesn't support ad hoc queries, and you can't use it for real.time properties (only transactional state).

Who and why? People try and give up on all technologies all of the time. We also find many people who try it and would never go back. You say you don't like it, have you tried it?

Quote:Original post by Codeka

Quote:Original post by jeffpk
One last comment. I've already invited anyone who wants to, to stop listening to the FUD

What FUD, specifically, are you referring to? The fact that you can't do ad-hoc queries on a Darkstar database? The fact that it's not suited to 30+ transactions per second per entity? Or something else? I don't think anybody's said anything else - sure, you might argue that you can make a game without those things, but that doesn't change the fact that those are limitations of the system.

And 30 transactions per entity, per second, is fine for FPS, not so much for MMOs where world state is much more important for solidity of the experience.

Quote:I don't want to make an ad hominem argument here, but I'm personally more inclined to trust the advice of hplus0603 - a longstanding member of this site with a proven track record of knowledgable and informed opinions - over the advice of some guy who only signed up yesterday to post in this one thread. Especially when those posts are dripping with attitude.

Well, to be fair, hplus0603 is the CTO for Foreterra, Inc., which he links to on his journal page here at GD.net, which had built a server system that, in many ways, competes with Darkstar based virtual worlds like Wonderland. So, I imagine his objectivity may be a bit biased.

Quote:I'm not saying that Darkstar is not worth investigating, just that it's not possible for it to be the silver bullet that some people seem to be advocating.

No one has the silver bullet for everything, including Darkstar, nor have we ever claimed it is. Darkstar solves some very complex issues regarding transaction integrity, something that will become more and more valuable over time in the online game space, load balancing and management. It is one piece of the puzzle, not the whole answer.

I would kindly suggest that you at least read the forums section at www.projectdarkstar.com to see what people are doing with the tech.

Quote:Original post by arbitus

Quote:Original post by ColinFitzgerald


I'm currently working on a MMO project (www.trzzonline.com) that uses Darkstar as a 'server'. After much research and deliberation PDS was clearly the best choice: It is an amazing system. What drew my attention to PDS was the fundamental design of the system: it solves the very hard problems for you and it does things 'correctly'. Overall it's too complex to summarize here.

And indeed, Darkstar does look like it could be a quality choice for indies and hobbyists. I think the "negativity" it is getting comes from experienced people who have either worked on systems that solved the problems Darkstar addressed long ago, or realize that you can achieve more by using a collection of other more robust libraries out there. I believe one of the features touted by Darkstar is its ease of use, no so much its robust application.

With that said, I would not consider Darkstar all that revolutionary, and I do believe that some of its core competencies are not exactly relevant anymore. "Zoning" and "sharding" are rarely a server-side limitation these days so much as a client-side limitation or a gameplay mechanic. For instance, zoning allows easier client side content management to allow for much better client performance as well as simpler client-side game architecture. Sharding is often use to disperse players across content rather than as a hard cap on connections to a single shard. These are both problems that Darkstar cannot solve as they are gameplay and client-side issues.

But, as with all open source projects, there is very real potential for a breakthrough using Darkstar, and I do not see any reason why a beginner/hobbyist/indie should not use it just like any other library to provide server-side components for their game.

And what is with the weird white-knighting of strange topics over the past few days here?

Edit: In addition, its database transactional model seems severely naive and a potential huge negative for those who are more experienced in the area.

EDIT: I misunderstood your intent of the comment in your edit. Sorry about that :)

[Edited by - CMelissinos on September 11, 2009 3:12:09 PM]

In the interests of transparency, especially where it comes to issues of potential bias, it's necessary for me as moderator to point out that CMelissinos has a Sun.com email address. (Not that this has been concealed - just that someone might miss the 'nor have we ever claimed it is' statement that suggests this.)

If we can stick to facts in the rest of the thread however, then personalities, reputations, and vested interests should become irrelevant.

Quote:Original post by Kylotan
In the interests of transparency, especially where it comes to issues of potential bias, it's necessary for me as moderator to point out that CMelissinos has a Sun.com email address. (Not that this has been concealed - just that someone might miss the 'nor have we ever claimed it is' statement that suggests this.)

If we can stick to facts in the rest of the thread however, then personalities, reputations, and vested interests should become irrelevant.

Oh, of course! I don't hide that fact. I am one of the people who started the project to begin with. However, I am not in here speaking out of turn with regard to other technologies at all.

As I said before, there is no silver bullet in ANY technology. However, hplus0603 is making statements about the technology, that others are taking as truth due to his status as a moderator, without disclosing that he is invested in technology that plays in the same space to some degree. As well, he has not stated if he has actually TRIED the technology before making some gross mis-statements about it. I have not tried Forterra's product, so would never make product/performance claims in a thread like this, with statements like "So I've heard...blah blah blah"

Yep, I'm on the Darkstar team, worked for Sun Microsystems for over 15 years now and am the Chief Gaming Officer there. Now, back to the discussion :)

Going back to talking as a regular poster:

Re: this comment "Again, hang a SQL server off the back end for long term storage and retrieval" and the previous comment from a different poster, "It uses Java serialization".

Does this mean you are essentially 'limited' to using Java for your game to get the full benefit from it? This is a pertinent question because the front page at www.projectdarkstar.com makes no mention of Java. The FAQ says that 'On the server side, Project Darkstar applications must be delivered as Java Byte Code', so does this essentially mean anything that runs on the Java VM will do?

I also don't see how easy it would be to 'hang a SQL server off the back'. Are there packages to automatically hook into the schema that would be in use for persistence? Would you have to use something like Hibernate? How would it compare to a typical system that has quite an explicit object -> table mapping which is automatically easy to query? Or are you talking about duplicating the data to a separate database for this purpose?

I realise some of you Darkstar guys want to say, "read the docs", "try it for yourself" but since that's not practical in every case for hobbyist and independent developers it would be good to get a high level view of the benefits it offers. With all due respect, several of the PDFs on the site are written as much to persuade as to inform and it would be useful to get a very high level view of the real basis of the system. If there are direct links that answer the questions and doubts though, feel free to post them.

Edit: Really, I was going to walk away. There is nothing more frustrating then trying to explain something to a student who doesn't want to learn. But Chris asked me to come back to talk to this question...

Quote:Original post by Kylotan
Going back to talking as a regular poster:

Re: this comment "Again, hang a SQL server off the back end for long term storage and retrieval" and the previous comment from a different poster, "It uses Java serialization".

Does this mean you are essentially 'limited' to using Java for your game to get the full benefit from it? This is a pertinent question because the front page at www.projectdarkstar.com makes no mention of Java. The FAQ says that 'On the server side, Project Darkstar applications must be delivered as Java Byte Code', so does this essentially mean anything that runs on the Java VM will do?

Yes, for your server app you can use any language that generates Java VM byte code, with the one caveat that there are a very few thing you can do in Java byte code that you cannot do in a Darkstar app. These are things that are tied to the VM instance. Since, in the multi-node case, Darkstar dynamically executes your code across a machine room full of VMs these wont correctly follow your code. One good example are static fields, which are actually fields on the Class structure in the VM.

For instance we have a few people already doing some work in Scala on the PDS.

Java serialization is actually properly "Java VM Serialization." Its a feature of the Java VM and is automatically supported by any language that runs on the VM.


Edit: To be clear, For your client, you can use just about anything. The communication protocol is both open and pluggable. We've seen clients written in Java, C, C++, C#, Python and AS3. And those are just the ones we know about. The Java and C API implementations are provided by the Sun Labs PDS team. The other implementations have all been written by the community.

Quote:
I also don't see how easy it would be to 'hang a SQL server off the back'. Are there packages to automatically hook into the schema that would be in use for persistence?

Funyn thing, this is going to be EXACTLY what I am talking about next week. Rebel Monkey's "Monkey Wrench" platform was a combination of the PDS with traditional web technologies (Glassfish, memcached, MySql). The short answer is that Darkstar Services are pluggable and extensible and a service can do anything normal code can do, including calling JDBC, web services, or your favorite Java to DB binding system.

The long answer, which is ALL the details of how we built a unified, single-signon platform that used the PDS for all game logic and a web system for all user profile, avatar editing and social features is a 50 min talk. I'm not allowed to post the slides until after I give the talk, but then I will be posting them up to the PDS community site.

Edit:

Quote:I realise some of you Darkstar guys want to say, "read the docs", "try it for yourself

Actually what we say is 'come join our community.' The project darkstar forums are quite active and the entire PDS research team reads them, answers questions and addresses issues that arise. The PDS is *very* much a community effort and the team sees community interaction as a very important part of that.

But it is hard for us (the team and the community) to be equally responsive to every other forum where questions might arise.

[Edited by - jeffpk on September 11, 2009 10:57:22 AM]