Sign in to follow this  

Stackless python

This topic is 4094 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Anyone here has embedded stackless python with C++ code especially a game engine? We are planning on embedding a scripting language with our engine and the obvious choices are Lua and Stackless Python (since we plan to use micro-threads). I saw a good article here We have guys (including myself) who are experienced with python so that would be a added advantage of using stackless. Is stackless python same as python programming language? Does that mean we can use many of the libraries available for python with stackless? That would be really exciting. I know Lua is time-tested and used by many game engines as the scripting language of choice. So at the end of the day is it better to play safe and use Lua or try something innovative and use Stackless. Thanks

Share this post


Link to post
Share on other sites
Stackless Python only changes some things about the internal workings of Python, so it should retain compatibility with everything that standard CPython can use. It's an interesting concept, but it doesn't scale well. I tried using it for my sim with 10000+ actors with a tasklet for each one, and found that the memory overhead was several megabytes. It was also about 25% slower than an iterative approach where I called a method on each object in succession. I'd say if you never plan to have more than 1000 tasklets, it's definitely something worth looking at.

Share this post


Link to post
Share on other sites
Quote:
Original post by drakostarI tried using it for my sim with 10000+ actors with a tasklet for each one, and found that the memory overhead was several megabytes. It was also about 25% slower than an iterative approach where I called a method on each object in succession.


Ah, measurements!
- What versions of Python and Stackless Python did you test?
- Memory overhead compared to what?
- What are the differences between the implementation with an "iterative approach" and the implementation with tasklets? They must be quite different: are they really doing the same work?

You made an interesting benchmark, but without explanations and analysis its relevance is unclear.
For example, did you prove that a tasklet costs several hundred bytes, without loading it with data? It can be an absurd amount of overhead or a reasonable amount of reserved empty memory or anything in between.

Share this post


Link to post
Share on other sites
I don't remember the exact versions, but it was the latest ones available on April 21, 2006, when I posted this on the OGRE forums:

=====================================================================
Sure enough, Stackless is about 25% slower than the usual iterative solution. Using 100,000 entities and doing ten iterations of some very simple calculations, Stackless starts showing its overhead.

Code:
stackless: 21.779
iterative: 17.263


In a similar test, I confirmed that it also has about 40MB of overhead.

I'm disappointed; I was hoping I could use Stackless in my project. But I think I might be able to fake some kind of cooperative scheduling without it. Something to work on when I have more time.

Anyway, here's the code I used:
Code:

from time import time
from random import randint

import stackless
from stackless import tasklet

class Group:
def __init__(self):
self.pos = (
randint(-1000, 1000),
randint(-1000, 1000) )

def run(self, dx):
self.pos = (
self.pos[0] + randint(-5, 5),
self.pos[1] + randint(-5, 5) )

class CoopGroup(Group):
def run(self, dx):
for i in range(10):
Group.run(self, dx)
stackless.schedule()

def test_stackless():
stackless.run()

def test_iterate(map):
for i in range(10):
for g in map:
g.run(0)

def main():
for n in range(100000):
g = CoopGroup()
t = tasklet(g.run)(0)
t = time()
test_stackless()
print "stackless:", (time() - t)

map = []
for n in range(100000):
g = Group()
map.append(g)
t = time()
test_iterate(map)
print "iterative:", (time() - t)

main()






=====================================================================

(...back in the present...)

Given this code, I think it's fairly obvious how to make a few modifications and check memory usage. Let me know if you think my analysis is flawed! It might be, and I admit I was testing some extreme conditions.

Running this exact code with Python 2.4.3 and Stackless 3.1b3 gives me:
Quote:
stackless: 23.4530000687
iterative: 18.25

Pretty much the same, Stackless takes a little more than 25% longer to execute. Just looking at Task Manager while it's executing, it takes 100MB and 55MB in each section, about 45MB overhead, since the data is the same, the only difference is the tasklets that have been created, about 470 bytes each. Of course this observation is very unscientific and may not matter at all. My big concern was always execution time, since I wanted simulation code that would run quickly even on older CPUs.

[Edited by - drakostar on September 21, 2006 5:14:28 AM]

Share this post


Link to post
Share on other sites
Quote:
Original post by daerid
I could be wrong, but isn't the whole point of Stackless to be faster?

Nope. Stackless means that it eliminates the usage of the C stack for keeping track of the program state, opting for its own graph instead. This allows for a very efficient (but of course not perfectly efficient) implementation of microthreads. It lets you write code in really interesting ways, and can be a perfect fit for scripting most game environments.

Share this post


Link to post
Share on other sites

This topic is 4094 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this