For Drew_Benton's question, that was also what I asked myself.

After I did research and tried weighing many options, as Washu already stated, the main point I am willing to use Stackless or PyPy is its concurrency mechanism. Also many documentations and presentations that EVE developers provided letting us to think out of our old technical models and implementations.

I am not going to build a giant realtime world as of EVE, but just some casual turn-based online games. So performance is not an issue yet, especially when EVE showed us they can greatly dealing with. GIL is not an issue too, my recent projects use multi-processes model and they work fine. So what light I see in [Stackless] Python is the nature of scripting language and the Actor model that will ease many issues I found in old days with C++. And yes, Python is capable with C/C++ extension as needed :-)


About GIL, since Python 2.6 they added multiprocessing package to help migrating thread-based to process-based concurrency and it can utilize multi-core CPU. However, be aware that serializing and communication between processes is expensive.



My still question, is there any network engine recommended for building an MMO?

Quote:Actually, the two end up being about the same in complexity probably. Writing an asynchronous erlang network server and a C++ one will encounter the same kinds of issues, and will end up having to solve them.

I agree that domain knowledge is essential when dealing with complex issues.

However, I very strongly disagree with your claim of equivalent complexity.

Erlang networking lets you work at a level of abstraction where most things that are issues (and complicated ones) in other languages have already been solved for you.

I assume that by asynchronous network server you mean one that uses nonblocking io (hence no need for thread-per-connection). Here's a network server in erlang with the following characteristics:

1) No thread-per-connection.

2) Uses advanced polling if environment supports it instead of plain select() (epoll on linux, kqueue on BSD, don't know if erlang can use windows IOCP).

3) Work done by each connection handler will scale to multiple cores (this is a consequence of the way the erlang scheduler works on multicore systems now) but the whole thing will only use MAX_CORES system threads to run.

The equivalent of the above is very tricky to write in C++, as it forces you to more or less directly deal with issues 2 and 3. Not in erlang; this server responds to "hi" messages and echoes messages it can't understand:

-module(async_server).-compile(export_all).-define(TCP_OPTIONS,[binary, {packet, 4}, {active, false}, {reuseaddr, true}]).-define(TIMEOUT, 500).init(Port) ->	{ok, Listen} = gen_tcp:listen(Port, ?TCP_OPTIONS),	loop(Listen).loop(Listen) ->	{ok, Socket} = gen_tcp:accept(Listen),	spawn_link(?MODULE, handle_conn, [Socket]),	loop(Listen).handle_conn(Socket) ->	case gen_tcp:recv(Socket, 0, ?TIMEOUT) of		{ok, <<"hi">>} ->			%do some work			gen_tcp:send(Socket, <<"hi urself">>);		{ok, Received} ->			%do some other work			gen_tcp:send(Socket, <<"can't understand ", Received/bytes>>);		{error, Reason} ->			error_logger:error_msg("Socket error ~p", [Reason])	end.	

It's pretty interesting that this snippet of code basically solves this nasty old problem out of the box, and is pretty trivial to extend to limit number of users, traffic-per-user, and other traffic shaping.

So I'd have to say choice of language for network programming can be extremely significant, at least to me; there are many projects that I'd write in erlang in an afternoon and that I wouldn't even want to attempt in C or python (generally those where performance is important, as that's the requirement that makes C and python networking ugly).

You can get an approximation of that ease, without the multicore scaling, in stackless python if you're willing to use the socket event wrapper contrib modules (stacklesssocket.py and socketlibevent.py) but I've been less than satisfied with them, ymmv (apart from the fact that they're not part of core stackless for some reason).

I have a question regarding Erlang.

Can I just use Erlang as a networking and server platform, and write everything else in Python or C++ running on top of Erlang servers?

Quote:Original post by Anonymous P

The equivalent of the above is very tricky to write in C++, as it forces you to more or less directly deal with issues 2 and 3. Not in erlang; this server responds to "hi" messages and echoes messages it can't understand:

I beg to differ.

While C++ is somewhat more verbose as language, developing this type of networking model is no longer hard.

C# comes with similar model as well.

Erlang has strong points, but this particular example is not necessarily the defining advantage. It also depends on type of networking and exact performance characteristics. And obviously, C++ will always lose due to verbosity, language nonsense, build process, ....

Quote:Original post by PaePae
I have a question regarding Erlang.

Can I just use Erlang as a networking and server platform, and write everything else in Python or C++ running on top of Erlang servers?

That's kind of the idea I had in the back of my mind when I made this post, Do you think this is a feasible server architecture design? (honestly can't believe I had a typo in the title and missed it), except I based my thinking along doing it in C++ simply because I know the language well enough to relate any answers I got about the idea. I personally don't know much about erlang other than the really simple programs I've tried, so trying not to bite off too much to chew at once, I just stuck with IOCP/C++.

I think the problem you will run into will be, which was pointed out to me more or less, the networking layer that handles the data processing isn't the problem that needs to be solved. I kind of see that point now after doing some more thinking about it. Let's say for a second you do have amazing network data processing servers that handles all that stuff, what about your game's logic servers and all those interactions?

If anything, you need your entire backend system to run on that platform (Erlang, in this case) to be able to really benefit from the language features. So, the problem as explanation to me is not of "How do I handle a lot of connections and data" as much as "How do you design your game to run in a distributed manner and not have the entire thing self-destruct?"

That's basically what I'm trying to figure out with the Python/MMO option because right now I am already looking into using Lua specifically for the benefit of being able to rapidly prototype out a MMO faster than I ever could in a non-scripting language. In my case, I can and want to sacrifice any performance possible as long as I am able to prototype out the design and really get an idea of "how things work".

For me at least, it's not about easy or hard or how much time something takes as much as just, I have no idea how things work, I want to learn and figure them out first hand. I have ideas of how things work, but it seems to be turning out they aren't so accurate. So, I'm trying anything I can think of and get advice on to build up the domain knowledge required to actually successfully implement such a solution. I've not read up on EVE's information they have released about their development process yet though, but that's certainly on the todo list.

The thing I was trying to figure out with my last post was, by using Stackless Python, will I actually be able to work through prototyping out a MMO using the language features it provides, or do I just not know enough yet to where there's no real benefit to me using it over any other scripting language for the task?

Of course, the answer to that in my case is that I should just try it myself after learning the Python language first, since trying to implement such a project in a language you do not fully understand will come around and bite you, as Washu so nicely put it. [wink]

Quote:Original post by hplus0603
Regarding the question: Does it help to use the platform, or do you need to know what you're doing? Sadly, you need to know what you're doing. The platforms are just various ways of organizing what you're doing, not ways of doing things per se.

The platform to me says: We understand the problem, and we solved it, so you don't have to. Or, more commonly - the cost of our platform is lower than cost of you learning and developing from scratch.

Some vendors deliver on that promise. Many more don't, and rely on quite realistic chance that users will never push the limits, or even come close to them.

Too frequently, this accidental success leads to hype following that becomes self-fulfilling prophecy, until only years later users realize other options are better.

And here the catch-22 occurs. To understand which vendor actually delivers, one must understand the problem.

Quote:Original post by hplus0603From what I understand, Stackless Python isn't actually "stackless" per se, but instead it moves parts of what used to be on the C stack onto an internal stack, and then it uses fibers for the C parts.

This is so wrong I barely understand it. Stackless Python does not use fibers. It just manipulates the stack. But yes, it is not "stackless", not any more at least.

Quote:What I don't understand is whether Stackless gets rid of the global Python interpreter lock, or whether that's still a scalability problem? Would Stackless not have an internal data structure problem if two threads tried to update the same dict at the same time?

Stackless as a fork of Python does not aim to change it, merely to extend it. Primarily with green threading, but also with the coroutine equivalent (can block the whole call stack, C frames included) channel communication mechanism and last but not least the ability to persist running code via pickling.

As such, there are no wacky changes like removing the GIL. It should be 100% compatible with existing code for standard Python, this includes both Python and C extensions.

Quote:Original post by WashuThe networking support of stackless is the same as that of Python, which is good enough for a reasonable sized game. You will find advantages though in moving to a more ASIO (IOCP for instance) setup, as EvE did not too long ago.

Yes, non-blocking IO is essential if anyone is going to make best use of Stackless. The "stacklesssocket" module I wrote can be googled, or if IOCP (on Windows) is desired, I've written the main part of an IOCP version at this Google hosted project.

Quote:Original post by hplus0603Thus, I would disagree with the recommendation that stackless is great for an MMO design that you start today. Note that EVE was started something like 8(?) years ago. In the future, you'll have to do more than what EVE is doing to be competetive, and they're already doing all they can with the current Python infrastructure.

They are?

I'd point out Second Life. They've been quite forthcoming with details about how they've implemented their scripting language and achieved scalability. That is, the ability to migrate running code from one node to another.

This is something that is possible with Stackless as well, and CCP are not using it at this time.