Embedding Python into prexisting game engine

Started by
9 comments, last by Kylotan 15 years, 10 months ago
I don't have that much experience in python but before I start delving into it I just wanted to see if what I want it to do is possible. I currently have most of a game engine coded in C++ as seperate classes (one for graphics, sound, levels, etc) My idea of how it could work is that a c++ program starts up and does all its initialization things and then starts up the respective classes (each one has a initialization function). The program would then call a script that would tell the engine to load the level and then play it, where the c++ engine would take over and play the level until it finishes and it would return a number to the script and the script would decide where to go from there (like game over or the next level). I like python, but I would be willing to use pretty much any other language to do this. The biggest problem that I see is there are lots of ways to export classes to python, but I want python to interact with a allocated class. I would also eventually like to have my AI coded in a scripting language, but I figure I should get this running first Thanks for all your help
Advertisement
I also tried to embed python, unfortunately you will have a hard time running several scripts independently from each other

I had no luck and switched to LUA instead.

It is pretty easy to embed and within a few minutes.


just search for "c++ embedding lua" on google and you will find some nice tutorials to get you started

also read the documentation about how to work with the "lua stack"
http://www.8ung.at/basiror/theironcross.html
I was going to use stackless python, which from my understanding should enable to to run several scripts, but I may try Lua as it seems more like it is designed to do what I want it to do so thanks.

If anyone else has a way of making a running program's variables accessible to python, I would enjoy to hear it as well.

Thanks again
Quote:Original post by Basiror
I also tried to embed python, unfortunately you will have a hard time running several scripts independently from each other

Why?
      //using boost::python      object main = import("__main__");      object global(main.attr("__dict__"));      dict namespace1;      dict namespace2;        exec("x=1",namespace1,namespace1);      exec("x=2",namespace2,namespace2);            std::cout << extract<int>(eval("x",namespace1,namespace1)) << std::endl;      std::cout << extract<int>(eval("x",namespace2,namespace2)) << std::endl;

output:
12


Aren't the scripts "x=1" and "x=2" running independently? Do you need more than this?
Take a look at these two links

http://www.boost.org/doc/libs/1_35_0/libs/python/doc/index.html
http://wiki.python.org/moin/boost.python

You will have to expose your c++ classes to python by writing the exact same as if you're extending python with a module, and then you'll be able to make instances available to the internal interpreter.

class Creature{    public:        int getStrength() { return m_strength; }    private:        int m_strength;};BOOST_PYTHON_MODULE(test){    class_<Creature>("Creature")        .def("getStrength", &Creature::getStrength)}


Let's say you have a function that does some calculations on a Creature object, and you want this functionality to be in a python script. I would do like this:

float doCalculations(const Creature& creature){    m_python->main_namespace["creature"] = creature;    m_python->executeScript("calcLibido.py");    return m_python->extract<float>("result");}


I have ommited exception handling and other safety measures, but the important issue here is that the script you call (calcLibido.py) must know about the creature variable, and put the result of its calculations in a result variable (float).

Other stuff to think about is that when this all goes out of scope, the python interpreter will garbage collect the reference and kill your c++ object... So I'd recommend wrapping this in boost::shared_ptr or maybe look into if it is possible to define some sort of policy in BOOST_PYTHON_MODULE.

m_python->main_namespace
is a boost::python::object of type dictionary. You should keep all this in a PythonEngine class or similar.

object main_module = import("__main__");object main_namespace = main_module.attr("__dict__");


If you need some code, I could extract parts from a project of mine and e-mail it to you. But I will need a couple of days to find the time. Sorry.

Hope that helps.

Christer.
Thanks for all the help, I found this page esspesially helpful: http://wiki.python.org/moin/boost.python/HowTo

For this project I will probably use lua, but I think I understand how to use python for this in the future

Again thanks.
Lua isn't that easy to use without a nice wrapper, you may want to look into luabind and tolua++ although both projects seem to be inactive since 2006. Last time I tried to use luabind I had to perform some minor changes in order to get it compiled with lua 5.1.2 but it worked well.
There are also some other wrappers, but I have never used them:
http://lua-users.org/wiki/LuaAddons
Quote:Original post by Kambiz
Quote:Original post by Basiror
I also tried to embed python, unfortunately you will have a hard time running several scripts independently from each other

Why?
*** Source Snippet Removed ***
output:
12


Aren't the scripts "x=1" and "x=2" running independently? Do you need more than this?


Did you ever use the Python C api with the python_newinterpreter and did you ever try to run it in completely independent threads?

You always have to swap dictionaries, which is just a hack not a solution, but you can t run scripts completely independent of each other such that you can run them in multiple threads simultanuously.

Theoretically it should be possible, but there seem to be a lot of problems arising when implementing it this way.
The way they posted it in their documentation did not work, I tried this out just a few weeks ago.

see this:
http://mail.python.org/pipermail/c++-sig/2007-April/012246.html
and this:
http://docs.python.org/api/threads.html

I also tried the following article with no luck, so I can t advise people to use python, they should just stick with something that is working out of the box, as lua does.

http://www.linuxjournal.com/article/3641


Quote:Original post by Kambiz
Lua isn't that easy to use without a nice wrapper, you may want to look into luabind and tolua++ although both projects seem to be inactive since 2006. Last time I tried to use luabind I had to perform some minor changes in order to get it compiled with lua 5.1.2 but it worked well.
There are also some other wrappers, but I have never used them:
http://lua-users.org/wiki/LuaAddons


I find the stack interface quite intuitive, but must agree that templated wrappers make function registering and the like easier to read.

[Edited by - Basiror on June 14, 2008 5:26:16 PM]
http://www.8ung.at/basiror/theironcross.html
Quote:Original post by Basiror
Did you ever use the Python C api with the python_newinterpreter and did you ever try to run it in completely independent threads?

A problem is as hard as the easies solution. boost::python is an awesome library and I don't see any reason not to use it.
I have run scripts in different threads. I just start the threads in a python script and let the interpreter manage everything:
// the global namesapce holds lock objects like// stdout_lock = threading.Lock() and the function// run is defined as follows:// def run(f) : execfile(f)object thread1 = eval("threading.Thread(target=run,args=('script1.py',))",global,namespace1);thread1.attr("start")();object thread2 = eval("threading.Thread(target=run,args=('script2.py',))",global,namespace2);thread2.attr("start")();thread1.attr("join")();thread2.attr("join")();

I haven't notices any issues until now.
There isn't anything wrong with lua, I simply prefer python (when speed doesn't matter) and I hadn't any problems embedding the interpreter in c++. The above solution may be considered as a hack or workaround but it works.

[Edited by - Kambiz on June 14, 2008 4:51:47 PM]
I don't think it is really difficult to embed Python in a C++ engine.

Swig or Boost are here to do the painful part for you ( I did it with Swig, and it's all right. Although I think boost is cleaner. )

There is a great tutorial from Sicrane that explains all you need :
http://members.gamedev.net/sicrane/articles/EmbeddingPythonPart1.html


On the thread part, Basiror, you are quite right : the 'GIL' implementation make it quite hard ( is it even possible ? ) to use multi threads on multi cores.

Now it depends on your needs.
I'm using Python to simplify my programmation, and make my progress quicker, so i just don't want multi thread in there ( although I use generators, and will perhaps jump into the stackless world if I get the time to ).

In my own personnal python driven engine, I don't use multi core at all, I essentially make 2D games/protos with it.

At work, with our next-gen-super-modern engine, ( with a custom script engine, but the problem would be the same), the AI is still on only one core, and the rest of the game is on the other cores ( display, physic, sound, pathfind, animation, ... ), so the engine fully uses all the cores we have, but there is no multithreads on the AI guys, that have enough problem on their own.

if you really need realy multi cores / multi thread on the script part, just don't use Python ( or use Jython or Iron Python ) !

Hope it helps,

Emmanuel

This topic is closed to new replies.

Advertisement