Sign in to follow this  
mrbastard

Python interprocess communication options?

Recommended Posts

I'm trying to get a handle on IPC options for Python. Background: I have a little app that generates simple GUIs for arbitrary classes using introspection. I'd like to be able to use it from other apps with minimal changes to the host app. The closest thing I can think of to compare it to is The Image Debugger, crossed with SmallTalk's inspector. Something like AutoGizmo( myClassInstance, options ) in the host app should open an instance of the GUI app. The apps then exchange data in real time. The host app is required to call an AutoGizmo.update function periodically, where the current values will be transferred to the GUI app. Similarly the GUI app needs to send changes to values back to the host app. I've pretty much decided against threading: the GIL seems to make using Python threads fairly pointless while still requiring extreme care to avoid errors. (Please say so if this assumption is incorrect!) The little GUI app uses wxPython, and it also seems much easier not to try and coerce wxPython's event loop to play nicely with the host app. The main use for this is with interactive apps BTW - stuff written in pygame and similar. I've been looking at the subprocess module, but as I know fairly little about IPC in general I'm finding it hard to work out what's supported on which platforms. Windows is my main platform so much of the posix stuff is out (no cygwin either thanks!). I'd also rather not use something windows-specific. So, it seems my choices are: -subprocess and use stdin stdout pipes for communication between processes -async_chat or similar to roll my own IPC based on TCP/IP sockets, possibly using pickle instead of serialization -something more heavyweight like XML-RPC - seems overkill, but already exists -something else? The Image Debugger uses memory mapped files - any use here? platform specific? thanks for any suggestions offered!

Share this post


Link to post
Share on other sites
If this was me I'd probably just open a TCP/IP socket and send pickled data. The asynchat approach is reasonable, though with just one socket I'd be tempted to thread that instead and push/pull the messages via queue.queue (so good they named it twice).

On the other hand I seem to recall you're a fan of the RMI approach: you might like Pyro (PYthon Remote Objects) instead! A quick look at the example looks promising.

Share this post


Link to post
Share on other sites
Thanks Kylotan.

Truly, my love of RMI knows no bounds [smile].

I'd heard of Pyro but not looked closely at it. The example does indeed look very promising. It seems a shame to require the user programmer to 'register' classes by deriving from Pyro.core.ObjBase. Though worthwhile for full 'distributed objects', I'd rather not lose the magic of having AutoGizmo work on arbitrary objects. I intend to use it regularly with classes exposed to python through boost.python. So I'd have to write some sort of adapter.

I suspect I'd end up with a Pyro.core.ObjBase derivitive with two methods: invokeMethodWithoutParams(name) and setMemberValue(name, value). Kind of defeating the whole object!

Alternately I could create classes on the fly which inherit from both Pyro.core.ObjBase and the host object type, then assign an instance of this new class the value of the host object. I'll have to read up on how Python handles assignment between base and derived classes in the face of multiple inheritance but I suspect things would get messy.

Pyro also seems to want an infinite loop, which I guess would mean using threading and queue.queue (or something) anyway.

I'm starting to favour TCP/IP and pickle as it seems likely to involve a similar amount of work, and the things I'd learn would be much more reuseable.

thanks again

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this