Quote:Original post by Drew_Benton
Here's my question on the whole issue about using Python for a MMO or even a MMO emulated server (more in my interests atm):

Can we, the average network programming Joes and Janes, actually come up with such a system that resembles (a smaller scale of) EVE's on our own by simply using the language itself or is there still a lot of special domain knowledge required?

For example, it is my understanding that in Erlang, concurrency, distributed processing, and fault tolerance mechanisms are an inherent part of the language in such a way that you just code your stuff and handle the specific events of when something goes wrong and everything is taken care of you due to how the language itself works. I know that's an oversimplification of it all, but just follow.

Whereas, in C++ when using Winsock, you have to specifically code and design an architecture that supports thread safe operations for concurrency, develop your own protocol for a distributed system, and work in your own external means for handling fault tolerance and process restarting (as hplus0603 gave a good example of what they did here. I know it's two different things being discussed, but just a loose example.)

So, do you actually need to know what you are doing before you do it when you use Stackless Python as the tool to implement a MMO, so that if you don't know what you are doing (yet), there's no real benefits to actually using it?

The real benefit that Stackless brings is the way it allows you to write more readable Python code. Got some logic you want to run? Launch it as a tasklet. Got something that looks remotely like a callback? Wrap it with a channel and have whatever invoked the logic that takes the callback look like a normal synchronous call.

Given you know about IO and that for the scheduler not to be blocked, non-blocking IO needs to be used (wrapped as described by channels) then you're pretty much set once that's taken care of.

You may have some idea that you'd like to use the tasklet pickling at some stage to build your own scalability solution, but that's something you can bring in in a way that best fits when the time is right.

I would say the answer is no.

Quote:Or, is it something that will allow you to come up with a solution that might be "good enough" on an Indie scale and you can generally just improve upon it as needed if you wanted to reach EVE levels?

This to me reads as a description of the development of EVE. Of course, Stackless is so much simpler these days being a stable product. Back when EVE was initially being developed Stackless had just come out, was actually "stackless", was then based on continuations and went through rewrites.

Quote:I ask because every time I see "Stackless Python", I see EVE mentioned, but nothing else. I would be inclined to think the former of my previous question and that people who are trying to learn the ins and outs of it all aren't going to benefit from using it because we simply don't have the experience required for a successful solution. Any thoughts on this?

There are other companies and projects out there using Stackless Python, however not all of them are interesting in discussing or divulging it.

Personally, I think people stick to the technologies that they know, trust and believe in. If they've bought into .NET, then they're going to find it hard to abandon it and jump over to something like Stackless. Same with C++ and other languages. I know there's one licensable MMO engine where the released EVE architecture papers were used to implement some of the systems without Stackless.

Quote:Original post by hplus0603

Quote:They've been quite forthcoming with details about how they've implemented their scripting language and achieved scalability.

Last I checked, Second Life claimed to support no more than 40 players per island (which maps to a server process).

Second Life are trying to compete directly with the OLIVE platform from Forterra Systems (where I work), so I won't say out loud what I think of Second Life claims about scalability, but I encourage you to do research based on all publicly available information, and make up your own mind.

I will note that, at the IETF forum on virtual world interoperability, several hard-core internet protocol architects expressed significant concern about the implementation of Second Life protocols with regards to correctness, security and scaling.

If you want to learn how to scale systems, I'd suggest you look at some systems that scale a lot higher than Second Life per instance, such as World of Warcraft, EVE Online, or perhaps upcoming games like MAG (256 players per "island"). Second Life largely scales by adding new instances (islands), which doesn't solve the problem of "more people want to meet in the same place."

You're responding to a selective sentence in my post, not the point I was trying to make. I do not care and have no interest in how well Second Life scales.

I was responding to your point about how CCP have used all that Python has to offer them and pointing out one functionality Second Life have implemented which CCP also has available through Stackless Python and do not use.

Nobody has commented on Twisted, so I suppose I will: don't bother. It looks like a sexy framework until you try to actually do things with it. Like, say, use SSL. Oops, the SSL handshake is a blocking operation, which *blocks everything*. A small hiccup in one client will result in the server freezing forever. Fiddling with threads is no help. Twisted doesn't even have multiprocessing support.

The fundamentals of Twisted, the very basic things it should be able to do half-decently, seem horrendously broken to me. You can't do nonblocking SSL? You don't use multiprocessing, which is absolutely critical for Python scalability thanks to the GIL? What? I wound up writing my own stuff using nothing but the excellent standard library, which as of 2.6 features more or less full OpenSSL support. It's just a little testing server to aid my client development, but it ended up with fewer lines of code and far better functionality than the version I had written with Twisted.

Honestly, in a few hours you can probably recreate the bits of Twisted you actually need (using little more than asyncore) and do it right.

Quote:Original post by PaePae
I have a question regarding Erlang.

Can I just use Erlang as a networking and server platform, and write everything else in Python or C++ running on top of Erlang servers?

Check out thrift.

Quote:Original post by drakostar
Nobody has commented on Twisted, so I suppose I will: don't bother. It looks like a sexy framework until you try to actually do things with it. Like, say, use SSL. Oops, the SSL handshake is a blocking operation, which *blocks everything*. A small hiccup in one client will result in the server freezing forever. Fiddling with threads is no help. Twisted doesn't even have multiprocessing support.

The fundamentals of Twisted, the very basic things it should be able to do half-decently, seem horrendously broken to me. You can't do nonblocking SSL? You don't use multiprocessing, which is absolutely critical for Python scalability thanks to the GIL? What? I wound up writing my own stuff using nothing but the excellent standard library, which as of 2.6 features more or less full OpenSSL support. It's just a little testing server to aid my client development, but it ended up with fewer lines of code and far better functionality than the version I had written with Twisted.

Honestly, in a few hours you can probably recreate the bits of Twisted you actually need (using little more than asyncore) and do it right.

I find it hard to believe that Twisted is this badly off, considering how much time and effort has gone into it. There are numerous companies out there that seem to be using it for various purposes, including C.C.P. Although it is used in C.C.P. for tool related purposes, rather than as the networking stack of the game itself. But then, Twisted wasn't in a suitable state when EVE development started, so who knows what might have happened.

As for using asyncore, having used it extensively, I consider the use of it to be a mistake, especially on Windows where it merely wraps the select module (which has a 512 connection limitation). To get best use out of it, you need to write your own framework around the asyncore framework, where it would be clearer if you just wrote your own framework to begin with.

Quote:Original post by rmtew
I find it hard to believe that Twisted is this badly off, considering how much time and effort has gone into it.

I'm completely open to anyone proving me wrong by giving me a simple SSL server example (with cert verification) that I can't break in five seconds.

As for a lack of multiprocessing support, this can easily be confirmed.

Quote:I consider the use of it to be a mistake, especially on Windows where it merely wraps the select module (which has a 512 connection limitation).

Who in their right mind would choose Windows as a game server? If nothing else, it's a waste of money.

asyncore is just fine for a lot of use cases, and it's dead simple. Premature optimization...etc. And if you do find that select() is the bottleneck, there are only a few short functions (asyncore is a Python module) you'd need to override to swap in epoll or kqueue instead.

Quote:Original post by drakostarWho in their right mind would choose Windows as a game server? If nothing else, it's a waste of money.

Yeah, well, you know, that's just, like, your opinion, man.

Quote:asyncore is just fine for a lot of use cases, and it's dead simple. Premature optimization...etc. And if you do find that select() is the bottleneck, there are only a few short functions (asyncore is a Python module) you'd need to override to swap in epoll or kqueue instead.

Asyncore is an arbitrary framework, the only thing it has going for it over and above other comparable solutions is that it is in the standard library.

As for your select solutions, given they do not apply to Windows, they are not really practical.