Quote:Original post by cg123
I have a bit of experience with Stackless Python - the fellow responsible for it, Christian Tismer, is a coworker of mine. Stackless Python is an excellent choice for MMO development. If you've heard of EVE Online, that was written entirely in Stackless.

Not entirely in stackless. Components of it are in stackless though. Others are in C.

The networking support of stackless is the same as that of Python, which is good enough for a reasonable sized game. You will find advantages though in moving to a more ASIO (IOCP for instance) setup, as EvE did not too long ago.

From what I understand, Stackless Python isn't actually "stackless" per se, but instead it moves parts of what used to be on the C stack onto an internal stack, and then it uses fibers for the C parts.

What I don't understand is whether Stackless gets rid of the global Python interpreter lock, or whether that's still a scalability problem? Would Stackless not have an internal data structure problem if two threads tried to update the same dict at the same time?

Quote:Original post by hplus0603
From what I understand, Stackless Python isn't actually "stackless" per se, but instead it moves parts of what used to be on the C stack onto an internal stack, and then it uses fibers for the C parts.

What I don't understand is whether Stackless gets rid of the global Python interpreter lock, or whether that's still a scalability problem? Would Stackless not have an internal data structure problem if two threads tried to update the same dict at the same time?

The GIL is still in place. While that doesn't stop you from having your C parts multi-threaded, it does mean the python parts wont be. There have been movements to refactor the GIL out of python in general, those have been resisted by morons who hate the idea of doing more than one thing at once. Refactoring the GIL out "is" possible, just not easy.

Yes, stackless uses fibers (on windows, something else on linux) to implement its stacklessness

The problem with the GIL is that it severely restricts in-process scalability across cores. Even five years ago, that wasn't much of a problem: "pizza box" servers were cheap and single threaded, and you would simply scale across processes instead. However, these days, I would not choose infrastructure that does not let me take advantage of multiple cores at the same time. This is my biggest beef with the modern Python ecosystem in general.

Thus, I would disagree with the recommendation that stackless is great for an MMO design that you start today. Note that EVE was started something like 8(?) years ago. In the future, you'll have to do more than what EVE is doing to be competetive, and they're already doing all they can with the current Python infrastructure.

Yes and no. You simply run multiple processes, or take advantage of ASIO/IOCP. Mind you, EvE is not entirely python, a large chunk of it is C/C++ and takes advantage of threading there, the python core is more used for tasklets, aka microprocesses.

Eve's been running fine with only minor hickups along the way, and their only real major issues have been the lack of IOCP usage, and certain traffic problems from their non-deterministic combat. That's with 40k people on the same cluster simultaneousnesly. Python will scale if you work with it. Their biggest slowdowns actually came from the number of transactions they were running on the SQL cluster, and on a lot of the heavy mathematics required for certain portions of the game. After switching to their new IOCP architecture and running ramsan's things are very responsive even in some of the most heavily loaded systems. Plus they can allocate an entire server to just a single cluster if need be in order to handle high processor issues (such as when two 500+ fleets engage in battle). This is something no other MMO has ever done (when was the last time you saw an MMO with 1+k people engaging in combat, along with between 5 (standard) - 20 (motherships/carriers) drones per person, or in other words, around 6k-21k elements requiring tracking, calculations, and feedback being passed around in realtime?

Quote:Original post by Drew_Benton
Here's my question on the whole issue about using Python for a MMO or even a MMO emulated server (more in my interests atm):

Can we, the average network programming Joes and Janes, actually come up with such a system that resembles (a smaller scale of) EVE's on our own by simply using the language itself or is there still a lot of special domain knowledge required?

No matter what, domain knowledge will be required. No platform, and certainly no language, will remove that requirement.

Quote:For example, it is my understanding that in Erlang, concurrency, distributed processing, and fault tolerance mechanisms are an inherent part of the language in such a way that you just code your stuff and handle the specific events of when something goes wrong and everything is taken care of you due to how the language itself works. I know that's an oversimplification of it all, but just follow.

If this were only true. Except its not. That gross oversimplification you mentioned... it's important. Those details are what ultimately make life interesting/difficult. It's not as simple as "I just write my code and it magically is concurrent and distributed!"

Quote:Whereas, in C++ when using Winsock, you have to specifically code and design an architecture that supports thread safe operations for concurrency, develop your own protocol for a distributed system, and work in your own external means for handling fault tolerance and process restarting (as hplus0603 gave a good example of what they did here. I know it's two different things being discussed, but just a loose example.)

Actually, the two end up being about the same in complexity probably. Writing an asynchronous erlang network server and a C++ one will encounter the same kinds of issues, and will end up having to solve them.

Quote:So, do you actually need to know what you are doing before you do it when you use Stackless Python as the tool to implement a MMO, so that if you don't know what you are doing (yet), there's no real benefits to actually using it? Or, is it something that will allow you to come up with a solution that might be "good enough" on an Indie scale and you can generally just improve upon it as needed if you wanted to reach EVE levels?

You're not going to magically be able to just go mmo = MakeMMO() and expect it to work. Even if there was some magical uber architecture that took care of that for you, you would still need to know a great deal about what you're doing. Stackless is just a tool, its a means of dealing with micro-processes, which can be used to simulate many things in a game world. EvE isn't all stackless, and large chunks of it are in C/C++ and do work OUTSIDE of the stackless simulation. But stackless is an important part of EvE because its ultimately what runs the game simulation.

Quote:I ask because every time I see "Stackless Python", I see EVE mentioned, but nothing else. I would be inclined to think the former of my previous question and that people who are trying to learn the ins and outs of it all aren't going to benefit from using it because we simply don't have the experience required for a successful solution. Any thoughts on this?

How many MMOs are out there now? How many of them are actually like EvE? I'll bet most MMOs use scripting on the backend in some form or another, just because it's the only way to reliably be able to change the game mechanics on the backend trivially. We just don't hear about it much.

EvE's use of python is brought up frequently in this arena simply because they've put a lot of technical knowledge out there on their use of python and how they have benefited from it (rapid prototyping, ease of use, and more). Most MMOs are very secretive about their architectures, they don't want you to know how they work on the backend. EvE practically has documentation on their site as to how the whole thing is put together.

[Edited by - Washu on September 13, 2009 1:48:51 PM]