Python: Extending/Embedding, libs in C/C++?

Started by
13 comments, last by Emmanuel77 16 years, 2 months ago
As some of my other posts here will attest, I'm currently learning Python. At the moment I'm still going through the basics, but I'd like to start thinking about which direction to take my learning in order to visualise where I want to go with respect to game development. In the future I'd like to incorporate Python into my 2D game system I've been working on for the last couple of years. Currently it's a mulch of useful utilities I built for simple 2D games in C++ and using SDL/OpenGL; I tend to refactor it every now and again when I have a new idea about how the whole thing should be structured. My desire for Python is for it to be used for the bulk of the game logic if possible, making it easier for me to put games together. I'm really only interested in working on 2D games; the most computationally intensive type of game I'd consider would be 2D games with a lot of physics or deeper than usual artificial intelligence. I'm not in the position to start using Python for this system yet, but I'd like to get an idea of the best way to use Python with external game code. There's two questions I'm unsure of: whether it's better to aim for embedding Python within the library or extending Python with the library; and if extending, whether to plan for using C++ or C for the code. From my understanding most commercial games out there today that use Python use it as an embedded scripting language. However I've read the article in favour of embedding here (Extending vs. Embedding: There is Only One Correct Decision), and it makes some good arguments. My gut feeling is that it's better conceptually to think of the whole program as being in Python with library calls for the computationally extensive parts, plus I can prototype the bulk of it in Python until performance becomes an issue. However there may be some downside to this that as a novice learning Python might not see. If extending, I'm also not sure whether it would be best to use C or C++ as my language of choice for the library functions. According to the Python documentation C appears to be preferred, and it seems to make more sense to me if I were writing smaller snippets of code to work with a majority Python program. But again, I'm not experienced in working with extending higher level programming languages like Python so I may be mistaken. I'm pretty comfortable with both languages (actually, I'm probably better with the C programing paradigms than C++). Are my gut feelings correct, and should I be seeing myself working on Python programs using extended 2D game library functions written in C (whether mine or from somewhere else?)
Advertisement
In all my messing around I have turned to favor extending. It makes things so much more flexible. For example, I have a module coded in C(++) on windows that handles all the graphics and sound. A friend of mine wanted to play my game on his mac, so what did I do? I reimplemented the same module API in pure python using pygame. Instant mac port without needing a mac. Imagine doing that if you embedded.

On C vs C++: all my extensions are C++ classes, and I use plain functions to wrap them into python, it's pretty easy once you do it once and then you can just cut and paste. Or you can use boost which has a pretty crazy set of python things.
I personally prefer embedding Python with C++. Basically I find when embedding Python it's easier to debug the C++ code and harder to debug the Python code compared to extending Python. However, debugging Python code is relatively painless when compared to debugging C++ code, and when debugging C++ code you really need all the help you can get.

There are downsides to this. In addition to the points in the article you mentioned, multiple threads and embedded Python do not mix well. On the other hand, I'm not sufficiently clever enough to debug multi-threaded code under the best of circumstances, so my applications are single-threaded anyways.

Also, the Python community isn't very friendly to people who embed rather than extend, so getting help from them can be an exercise in frustration if you embed.
Quote:Original post by wendigo23
In all my messing around I have turned to favor extending. It makes things so much more flexible. For example, I have a module coded in C(++) on windows that handles all the graphics and sound. A friend of mine wanted to play my game on his mac, so what did I do? I reimplemented the same module API in pure python using pygame. Instant mac port without needing a mac. Imagine doing that if you embedded.

Thanks for that; it's good to know that approach worked for you in practice. I might then plan on writing the whole thing in Python with pygame (or some other game library) first before considering rolling in my own functions.

Quote:On C vs C++: all my extensions are C++ classes, and I use plain functions to wrap them into python, it's pretty easy once you do it once and then you can just cut and paste. Or you can use boost which has a pretty crazy set of python things.

I've only glanced at the documentation for Boost's python links, so I got the impression it was mainly for embedding. If it's also for extending that would be good, although I'm not sure if I need all the extra functionality C++ provides if I'm just writing extension libraries for Python.
Quote:Original post by Trapper Zoid
I've only glanced at the documentation for Boost's python links, so I got the impression it was mainly for embedding. If it's also for extending that would be good, although I'm not sure if I need all the extra functionality C++ provides if I'm just writing extension libraries for Python.


Actually, it's mostly for extending. The basic documentation actually is missing a few details if you want to embed.
Quote:Original post by SiCrane
I personally prefer embedding Python with C++. Basically I find when embedding Python it's easier to debug the C++ code and harder to debug the Python code compared to extending Python. However, debugging Python code is relatively painless when compared to debugging C++ code, and when debugging C++ code you really need all the help you can get.

Is that because it's harder to debug the embedded stuff in isolation from the rest of the program? If I were to extend rather than embed, would it be wise to consider writing native C or C++ wrapper code for the purposes of unit testing?

Quote:There are downsides to this. In addition to the points in the article you mentioned, multiple threads and embedded Python do not mix well. On the other hand, I'm not sufficiently clever enough to debug multi-threaded code under the best of circumstances, so my applications are single-threaded anyways.

Me too; my apps haven't been advanced enough for me to worry about multi-threading. Consider everything I plan to do to be single-threaded for now.

Quote:Also, the Python community isn't very friendly to people who embed rather than extend, so getting help from them can be an exercise in frustration if you embed.

That sucks; is there a strain of elitism in the Python community?
I personally have only done work embedding with Boost.Python. I think really the only extra step is starting up the interpreter. Then depending on what you're doing you may need to call Python from C++.

Boost.Python's syntax is odd. When I first saw it I wondered how it even compiled properly. Its not too bad once you delve into it, but there are definitely a few gotchas.

You can also use the Python C API to link up your code, but something automagic is way better if you have a lot of code. There is also a generator for Boost.Python called Py++ that'll do all the binding code for you.
Quote:Original post by Trapper Zoid
Is that because it's harder to debug the embedded stuff in isolation from the rest of the program? If I were to extend rather than embed, would it be wise to consider writing native C or C++ wrapper code for the purposes of unit testing?

It's more of a combination of the fact because working with DLLs sucks in general and due to the way that I approach debugging. For example, leaving a Python instance open can keep the DLL you're working on loaded, so I'd constantly find that the linker wouldn't write to the file. People with different coding habits may find that debugging extension modules to be about the same.

For unit testing, unit tests in Python should be fine.
Quote:
That sucks; is there a strain of elitism in the Python community?

Try to find me a programming language community that doesn't have a strain of elitism.
Quote:Original post by Trapper Zoid
From my understanding most commercial games out there today that use Python use it as an embedded scripting language. However I've read the article in favour of embedding here (Extending vs. Embedding: There is Only One Correct Decision), and it makes some good arguments. My gut feeling is that it's better conceptually to think of the whole program as being in Python with library calls for the computationally extensive parts, plus I can prototype the bulk of it in Python until performance becomes an issue. However there may be some downside to this that as a novice learning Python might not see.


This article doesn't make any good arguments it is fluff at best and misleading at worst. Any programmer who is surprised or confused by python being embed in stead of extended is either not bright enough to call themselves a programmer or is surprised and confused by the use of python at all. I mean really can you name a programmer who can program in python but doesn't know C/C++? If you can find a few that have learned python and can't learn C then I don't think you want them working on your project.

I think and has been said previously embedding is the way to go at first, as it makes debugging easier which you are going to be doing a lot of when you first start learning the interop thing. In the end you are going to want to extend though as it leverages the rapid development abilities of python and lets you speed up the parts that need it.
Quote:Original post by SiCrane
That sucks; is there a strain of elitism in the Python community?

Try to find me a programming language community that doesn't have a strain of elitism.
True; I originally wrote that question as "is there a strain of elitism in the Python community (or at least, more than usual)?" but I edited it down. I guess the Holy Language Wars will continue to be waged for as long as programmers exist, regardless of language.

Thanks for the replies; it's helping me think ahead. Since I'm still new to Python it's probably best for me to continue working up the game development chain, then consider splicing in some library code once I reach a certain level of complexity.

This topic is closed to new replies.

Advertisement